[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:11:44.514461 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:33547
I20260709 12:11:44.516244 22107 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:11:44.517580 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:44.537084 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:44.537227 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:44.537257 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:45.739241 22115 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260709 12:11:45.739372 22107 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:11:45.743948 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:45.744190 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:45.744341 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599105744325 us; error 0 us; skew 500 ppm
I20260709 12:11:45.745157 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:45.751199 22107 webserver.cc:533] Webserver started at http://127.21.150.254:37771/ using document root <none> and password file <none>
I20260709 12:11:45.752056 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:45.752254 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:45.752661 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:45.758579 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "d3ccd713489c4d409eee5fdd03f3fae1"
format_stamp: "Formatted at 2026-07-09 12:11:45 on dist-test-slave-zctp"
I20260709 12:11:45.766844 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:11:45.772117 22122 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:45.773489 22107 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 12:11:45.773875 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "d3ccd713489c4d409eee5fdd03f3fae1"
format_stamp: "Formatted at 2026-07-09 12:11:45 on dist-test-slave-zctp"
I20260709 12:11:45.774230 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:45.826201 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:45.827788 22107 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:11:45.828280 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:45.903518 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:33547
I20260709 12:11:45.903599 22173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:33547 every 8 connection(s)
I20260709 12:11:45.911859 22174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:45.935009 22174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: Bootstrap starting.
I20260709 12:11:45.941249 22174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:45.943226 22174 log.cc:826] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: Log is configured to *not* fsync() on all Append() calls
I20260709 12:11:45.948969 22174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: No bootstrap required, opened a new log
I20260709 12:11:45.970286 22174 raft_consensus.cc:359] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER }
I20260709 12:11:45.970903 22174 raft_consensus.cc:385] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:45.971176 22174 raft_consensus.cc:740] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ccd713489c4d409eee5fdd03f3fae1, State: Initialized, Role: FOLLOWER
I20260709 12:11:45.972090 22174 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER }
I20260709 12:11:45.972608 22174 raft_consensus.cc:399] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:11:45.972922 22174 raft_consensus.cc:493] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:11:45.973280 22174 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:45.977604 22174 raft_consensus.cc:515] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER }
I20260709 12:11:45.978443 22174 leader_election.cc:304] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3ccd713489c4d409eee5fdd03f3fae1; no voters: 
I20260709 12:11:45.980278 22174 leader_election.cc:290] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:11:45.980640 22177 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:45.983068 22177 raft_consensus.cc:697] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 1 LEADER]: Becoming Leader. State: Replica: d3ccd713489c4d409eee5fdd03f3fae1, State: Running, Role: LEADER
I20260709 12:11:45.984130 22177 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER }
I20260709 12:11:45.985164 22174 sys_catalog.cc:565] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:11:45.995232 22178 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3ccd713489c4d409eee5fdd03f3fae1. Latest consensus state: current_term: 1 leader_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER } }
I20260709 12:11:45.996035 22178 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:45.995119 22179 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ccd713489c4d409eee5fdd03f3fae1" member_type: VOTER } }
I20260709 12:11:45.997396 22179 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:11:46.000257 22186 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:11:46.012650 22186 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:11:46.015136 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:11:46.032469 22186 catalog_manager.cc:1383] Generated new cluster ID: 1899fe902cc24c12a6bf946af197f759
I20260709 12:11:46.032775 22186 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:11:46.055617 22186 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:11:46.057147 22186 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:11:46.073156 22186 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: Generated new TSK 0
I20260709 12:11:46.074120 22186 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:11:46.082068 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:46.091274 22196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:46.092427 22197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:46.095934 22199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:46.096050 22107 server_base.cc:1061] running on GCE node
I20260709 12:11:46.097023 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:46.097299 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:46.097539 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599106097508 us; error 0 us; skew 500 ppm
I20260709 12:11:46.098351 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:46.102455 22107 webserver.cc:533] Webserver started at http://127.21.150.193:40641/ using document root <none> and password file <none>
I20260709 12:11:46.103168 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:46.103420 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:46.103767 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:46.105342 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.112449 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:11:46.117998 22204 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.119175 22107 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:11:46.119510 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.119837 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:46.153071 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:46.154489 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:46.157465 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:46.162084 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:46.162374 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.162664 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:46.162856 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.266984 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:45045
I20260709 12:11:46.267117 22266 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:45045 every 8 connection(s)
I20260709 12:11:46.273902 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:46.283772 22271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:46.285511 22272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:11:46.294538 22274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:46.296564 22107 server_base.cc:1061] running on GCE node
I20260709 12:11:46.297698 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:11:46.297979 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:46.298192 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599106298174 us; error 0 us; skew 500 ppm
I20260709 12:11:46.299002 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:46.299414 22267 heartbeater.cc:344] Connected to a master server at 127.21.150.254:33547
I20260709 12:11:46.299985 22267 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:46.301728 22267 heartbeater.cc:507] Master 127.21.150.254:33547 requested a full tablet report, sending...
I20260709 12:11:46.307212 22139 ts_manager.cc:194] Registered new tserver with Master: a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:11:46.308279 22107 webserver.cc:533] Webserver started at http://127.21.150.194:34995/ using document root <none> and password file <none>
I20260709 12:11:46.309019 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:46.309293 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:46.309665 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:46.310559 22139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40192
I20260709 12:11:46.311249 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "cd4d6e7b1323429c8880fe673141ca39"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.318967 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.003s
I20260709 12:11:46.323731 22279 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.324684 22107 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:11:46.324986 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "cd4d6e7b1323429c8880fe673141ca39"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.325268 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:46.345242 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:46.346591 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:46.349260 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:46.352725 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:46.352905 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.353192 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:46.353345 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.462294 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:37745
I20260709 12:11:46.462407 22341 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:37745 every 8 connection(s)
I20260709 12:11:46.470885 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:11:46.480280 22345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:46.484849 22342 heartbeater.cc:344] Connected to a master server at 127.21.150.254:33547
I20260709 12:11:46.485316 22342 heartbeater.cc:461] Registering TS with master...
W20260709 12:11:46.485592 22346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:46.486846 22342 heartbeater.cc:507] Master 127.21.150.254:33547 requested a full tablet report, sending...
W20260709 12:11:46.487638 22348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:11:46.488648 22107 server_base.cc:1061] running on GCE node
I20260709 12:11:46.489696 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:11:46.489660 22139 ts_manager.cc:194] Registered new tserver with Master: cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
W20260709 12:11:46.490022 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:11:46.490234 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599106490217 us; error 0 us; skew 500 ppm
I20260709 12:11:46.490939 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:11:46.491469 22139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40204
I20260709 12:11:46.495149 22107 webserver.cc:533] Webserver started at http://127.21.150.195:39827/ using document root <none> and password file <none>
I20260709 12:11:46.495648 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:11:46.495838 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:11:46.496110 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:11:46.497200 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "fc3e67a5db6c447583f943602f281536"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.501924 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:11:46.505288 22353 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.506137 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:11:46.506402 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "fc3e67a5db6c447583f943602f281536"
format_stamp: "Formatted at 2026-07-09 12:11:46 on dist-test-slave-zctp"
I20260709 12:11:46.506680 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:11:46.520183 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:11:46.521409 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:11:46.523239 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:11:46.526782 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:11:46.527045 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.527295 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:11:46.527529 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:11:46.624315 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:46245
I20260709 12:11:46.624857 22415 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:46245 every 8 connection(s)
I20260709 12:11:46.645401 22416 heartbeater.cc:344] Connected to a master server at 127.21.150.254:33547
I20260709 12:11:46.646011 22416 heartbeater.cc:461] Registering TS with master...
I20260709 12:11:46.646944 22416 heartbeater.cc:507] Master 127.21.150.254:33547 requested a full tablet report, sending...
I20260709 12:11:46.649298 22139 ts_manager.cc:194] Registered new tserver with Master: fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:46.650128 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018639168s
I20260709 12:11:46.650889 22139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40214
I20260709 12:11:46.704027 22139 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40230:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:11:46.707706 22139 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:11:46.800365 22381 tablet_service.cc:1511] Processing CreateTablet for tablet 7e59dba78ece4f629820bd43f26fd10a (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:46.802734 22381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e59dba78ece4f629820bd43f26fd10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.815248 22380 tablet_service.cc:1511] Processing CreateTablet for tablet 2211c5e3441e4641be59f669903d8a42 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:46.817345 22380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2211c5e3441e4641be59f669903d8a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.822410 22379 tablet_service.cc:1511] Processing CreateTablet for tablet 5a57670b155f42dabe1699c849e90e92 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:46.826181 22379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a57670b155f42dabe1699c849e90e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.829679 22378 tablet_service.cc:1511] Processing CreateTablet for tablet c19ef16b692341018afac81fbd8b24bc (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:46.831303 22378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19ef16b692341018afac81fbd8b24bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.848860 22307 tablet_service.cc:1511] Processing CreateTablet for tablet 7e59dba78ece4f629820bd43f26fd10a (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:46.850332 22307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e59dba78ece4f629820bd43f26fd10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.852850 22304 tablet_service.cc:1511] Processing CreateTablet for tablet c19ef16b692341018afac81fbd8b24bc (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:46.854332 22304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19ef16b692341018afac81fbd8b24bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.854843 22377 tablet_service.cc:1511] Processing CreateTablet for tablet 59269a8cb6ec4500b34f4e7292005694 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:46.856518 22377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59269a8cb6ec4500b34f4e7292005694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.860761 22303 tablet_service.cc:1511] Processing CreateTablet for tablet 59269a8cb6ec4500b34f4e7292005694 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:46.870700 22303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59269a8cb6ec4500b34f4e7292005694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.876382 22230 tablet_service.cc:1511] Processing CreateTablet for tablet 5a57670b155f42dabe1699c849e90e92 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:46.875814 22231 tablet_service.cc:1511] Processing CreateTablet for tablet 2211c5e3441e4641be59f669903d8a42 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:46.885581 22306 tablet_service.cc:1511] Processing CreateTablet for tablet 2211c5e3441e4641be59f669903d8a42 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:11:46.885686 22381 tablet_service.cc:1511] Processing CreateTablet for tablet 03c8c69d2f8c424799063791f4f92c1d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:46.888264 22306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2211c5e3441e4641be59f669903d8a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.891116 22230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a57670b155f42dabe1699c849e90e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.894608 22228 tablet_service.cc:1511] Processing CreateTablet for tablet 59269a8cb6ec4500b34f4e7292005694 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:11:46.893808 22381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c8c69d2f8c424799063791f4f92c1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.888959 22305 tablet_service.cc:1511] Processing CreateTablet for tablet 5a57670b155f42dabe1699c849e90e92 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:11:46.888967 22232 tablet_service.cc:1511] Processing CreateTablet for tablet 7e59dba78ece4f629820bd43f26fd10a (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:11:46.900303 22229 tablet_service.cc:1511] Processing CreateTablet for tablet c19ef16b692341018afac81fbd8b24bc (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:11:46.891088 22302 tablet_service.cc:1511] Processing CreateTablet for tablet 03c8c69d2f8c424799063791f4f92c1d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:46.909175 22434 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:46.915587 22231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2211c5e3441e4641be59f669903d8a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.910053 22227 tablet_service.cc:1511] Processing CreateTablet for tablet 03c8c69d2f8c424799063791f4f92c1d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:11:46.911067 22226 tablet_service.cc:1511] Processing CreateTablet for tablet a3468b676b534142851875be523c6ee0 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:46.914795 22301 tablet_service.cc:1511] Processing CreateTablet for tablet a3468b676b534142851875be523c6ee0 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:46.922156 22228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59269a8cb6ec4500b34f4e7292005694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.921288 22302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c8c69d2f8c424799063791f4f92c1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.928982 22225 tablet_service.cc:1511] Processing CreateTablet for tablet 148682670bd0400bbe7e8e3545f7937d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:46.933701 22300 tablet_service.cc:1511] Processing CreateTablet for tablet 148682670bd0400bbe7e8e3545f7937d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:46.932031 22305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a57670b155f42dabe1699c849e90e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.938652 22300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148682670bd0400bbe7e8e3545f7937d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.938160 22229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19ef16b692341018afac81fbd8b24bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.941560 22301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3468b676b534142851875be523c6ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.940469 22378 tablet_service.cc:1511] Processing CreateTablet for tablet a3468b676b534142851875be523c6ee0 (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:11:46.943670 22378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3468b676b534142851875be523c6ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.945976 22227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03c8c69d2f8c424799063791f4f92c1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:46.954444 22376 tablet_service.cc:1511] Processing CreateTablet for tablet 148682670bd0400bbe7e8e3545f7937d (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:11:46.980161 22225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148682670bd0400bbe7e8e3545f7937d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.013818 22226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3468b676b534142851875be523c6ee0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.025580 22232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e59dba78ece4f629820bd43f26fd10a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:11:47.041585 22187 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:47.040491 22376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148682670bd0400bbe7e8e3545f7937d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.025660 22224 tablet_service.cc:1511] Processing CreateTablet for tablet af9f742961e44504ab2db17c60a2730b (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:47.095937 22224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af9f742961e44504ab2db17c60a2730b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.137197 22304 tablet_service.cc:1511] Processing CreateTablet for tablet af9f742961e44504ab2db17c60a2730b (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:47.139037 22304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af9f742961e44504ab2db17c60a2730b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.139925 22435 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.198669 22434 tablet_bootstrap.cc:654] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.226246 22436 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.242710 22435 tablet_bootstrap.cc:654] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.227854 22378 tablet_service.cc:1511] Processing CreateTablet for tablet af9f742961e44504ab2db17c60a2730b (DEFAULT_TABLE table=test-workload [id=57289c3281ae4679bc96a0649b004ee5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:11:47.246482 22378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af9f742961e44504ab2db17c60a2730b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:11:47.243716 22436 tablet_bootstrap.cc:654] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.260210 22434 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.261420 22434 ts_tablet_manager.cc:1403] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.383s	user 0.015s	sys 0.016s
I20260709 12:11:47.266343 22434 raft_consensus.cc:359] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.267695 22434 raft_consensus.cc:385] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.268324 22434 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.269729 22435 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.271085 22435 ts_tablet_manager.cc:1403] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.217s	user 0.020s	sys 0.000s
I20260709 12:11:47.270624 22434 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.276602 22435 raft_consensus.cc:359] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.277464 22435 raft_consensus.cc:385] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.277899 22435 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.278829 22435 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.280596 22436 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.281080 22436 ts_tablet_manager.cc:1403] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.003s
I20260709 12:11:47.286159 22416 heartbeater.cc:499] Master 127.21.150.254:33547 was elected leader, sending a full tablet report...
I20260709 12:11:47.288897 22435 ts_tablet_manager.cc:1434] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 12:11:47.289678 22342 heartbeater.cc:499] Master 127.21.150.254:33547 was elected leader, sending a full tablet report...
I20260709 12:11:47.290723 22434 ts_tablet_manager.cc:1434] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:11:47.291394 22434 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.293841 22436 raft_consensus.cc:359] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.294584 22436 raft_consensus.cc:385] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.294924 22435 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.286288 22440 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.295791 22440 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.298977 22434 tablet_bootstrap.cc:654] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.294898 22436 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.299845 22436 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.300417 22440 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:11:47.304852 22435 tablet_bootstrap.cc:654] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.305459 22441 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.306285 22441 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.314843 22267 heartbeater.cc:499] Master 127.21.150.254:33547 was elected leader, sending a full tablet report...
I20260709 12:11:47.328533 22436 ts_tablet_manager.cc:1434] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.047s	user 0.011s	sys 0.008s
I20260709 12:11:47.329576 22436 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.330997 22441 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.346277 22436 tablet_bootstrap.cc:654] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.356726 22435 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.357851 22435 ts_tablet_manager.cc:1403] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.008s
I20260709 12:11:47.363631 22435 raft_consensus.cc:359] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.364389 22435 raft_consensus.cc:385] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.364709 22435 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.367224 22435 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.372089 22435 ts_tablet_manager.cc:1434] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 12:11:47.372408 22434 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.372917 22434 ts_tablet_manager.cc:1403] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.000s
I20260709 12:11:47.373311 22435 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
W20260709 12:11:47.384168 22417 tablet.cc:2406] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:11:47.380838 22434 raft_consensus.cc:359] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.387615 22434 raft_consensus.cc:385] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.389540 22434 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.390302 22434 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.401856 22436 tablet_bootstrap.cc:492] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.403043 22434 ts_tablet_manager.cc:1434] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 12:11:47.401197 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.402500 22436 ts_tablet_manager.cc:1403] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.073s	user 0.018s	sys 0.014s
I20260709 12:11:47.404104 22434 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
W20260709 12:11:47.407330 22280 leader_election.cc:343] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.408828 22435 tablet_bootstrap.cc:654] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.410286 22441 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.410750 22441 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.412876 22441 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:11:47.414501 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:47.415582 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.415895 22317 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 0.
I20260709 12:11:47.419471 22357 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
W20260709 12:11:47.419489 22280 leader_election.cc:343] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.422118 22440 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.422473 22440 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.423026 22440 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.422977 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.426685 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.427527 22436 raft_consensus.cc:359] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.428277 22436 raft_consensus.cc:385] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.426326 22391 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 0.
I20260709 12:11:47.429406 22436 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.438697 22281 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:47.440734 22441 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.441097 22441 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.441656 22441 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.431993 22436 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.446763 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.447520 22436 ts_tablet_manager.cc:1434] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.042s	user 0.002s	sys 0.006s
I20260709 12:11:47.453171 22440 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.454320 22436 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.455277 22440 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:11:47.455741 22390 raft_consensus.cc:2393] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd4d6e7b1323429c8880fe673141ca39 in current term 1: Already voted for candidate fc3e67a5db6c447583f943602f281536 in this term.
I20260709 12:11:47.448257 22434 tablet_bootstrap.cc:654] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
W20260709 12:11:47.448453 22354 leader_election.cc:343] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.450860 22441 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.460289 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:47.457702 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:47.461702 22317 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.463707 22441 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.464821 22435 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.465519 22435 ts_tablet_manager.cc:1403] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.092s	user 0.011s	sys 0.020s
W20260709 12:11:47.465993 22354 leader_election.cc:343] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.466059 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:47.466737 22242 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.465041 22281 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536
I20260709 12:11:47.468900 22435 raft_consensus.cc:359] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.469688 22435 raft_consensus.cc:385] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.469548 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:47.470021 22435 raft_consensus.cc:740] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.470219 22390 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.470809 22435 consensus_queue.cc:260] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.475910 22434 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.476397 22434 ts_tablet_manager.cc:1403] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.073s	user 0.006s	sys 0.008s
I20260709 12:11:47.477376 22242 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:47.477607 22390 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:47.478691 22280 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:47.479322 22434 raft_consensus.cc:359] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.480122 22434 raft_consensus.cc:385] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.480423 22434 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.481604 22317 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 1.
I20260709 12:11:47.482257 22441 raft_consensus.cc:2749] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.483028 22357 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:47.483906 22440 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:47.484382 22440 raft_consensus.cc:697] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:47.484939 22441 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:47.485245 22440 consensus_queue.cc:237] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.484272 22436 tablet_bootstrap.cc:654] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.485394 22441 raft_consensus.cc:697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:47.489167 22441 consensus_queue.cc:237] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.501515 22435 ts_tablet_manager.cc:1434] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.036s	user 0.011s	sys 0.003s
I20260709 12:11:47.502533 22435 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.481200 22434 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.513710 22434 ts_tablet_manager.cc:1434] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.001s
I20260709 12:11:47.514463 22434 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.514551 22137 catalog_manager.cc:5697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:47.520700 22434 tablet_bootstrap.cc:654] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.520188 22138 catalog_manager.cc:5697] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:47.523061 22435 tablet_bootstrap.cc:654] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.526492 22436 tablet_bootstrap.cc:492] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.526937 22436 ts_tablet_manager.cc:1403] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.073s	user 0.017s	sys 0.009s
I20260709 12:11:47.531819 22436 raft_consensus.cc:359] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.532536 22436 raft_consensus.cc:385] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.532824 22436 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.533845 22434 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.533731 22436 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:47.534354 22434 ts_tablet_manager.cc:1403] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 12:11:47.537626 22434 raft_consensus.cc:359] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.538559 22434 raft_consensus.cc:385] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.538897 22436 ts_tablet_manager.cc:1434] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.007s
I20260709 12:11:47.538945 22434 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.540067 22436 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.540123 22434 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.543180 22434 ts_tablet_manager.cc:1434] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:11:47.543694 22444 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.544185 22444 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.545531 22435 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.546053 22435 ts_tablet_manager.cc:1403] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.008s
I20260709 12:11:47.547194 22444 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.544158 22434 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.552502 22435 raft_consensus.cc:359] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.555140 22434 tablet_bootstrap.cc:654] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.560272 22435 raft_consensus.cc:385] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.561084 22436 tablet_bootstrap.cc:654] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.564766 22435 raft_consensus.cc:740] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.570351 22440 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.571024 22440 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.569638 22435 consensus_queue.cc:260] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.576409 22435 ts_tablet_manager.cc:1434] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.004s
I20260709 12:11:47.577497 22435 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.582677 22440 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.585414 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
W20260709 12:11:47.587215 22357 leader_election.cc:343] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.590157 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
W20260709 12:11:47.591753 22208 leader_election.cc:343] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.593114 22436 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.585695 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.593888 22436 ts_tablet_manager.cc:1403] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.054s	user 0.004s	sys 0.017s
W20260709 12:11:47.594942 22354 leader_election.cc:343] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.595386 22354 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:47.596266 22440 raft_consensus.cc:2749] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.598663 22440 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.599128 22440 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.600833 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.601408 22440 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.601538 22390 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:47.606741 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.607434 22242 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 0.
I20260709 12:11:47.607620 22434 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.602906 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:47.608754 22354 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:47.608237 22434 ts_tablet_manager.cc:1403] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.064s	user 0.021s	sys 0.011s
W20260709 12:11:47.609898 22357 leader_election.cc:343] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.606063 22206 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:47.610882 22440 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.611310 22440 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.611682 22440 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.611336 22444 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.614120 22444 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.613749 22434 raft_consensus.cc:359] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.614465 22444 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.614549 22434 raft_consensus.cc:385] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.614876 22434 raft_consensus.cc:740] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.618651 22435 tablet_bootstrap.cc:654] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.621306 22434 consensus_queue.cc:260] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.626240 22434 ts_tablet_manager.cc:1434] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:11:47.627318 22434 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.634291 22434 tablet_bootstrap.cc:654] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.606643 22436 raft_consensus.cc:359] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.634894 22436 raft_consensus.cc:385] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.635342 22436 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.636276 22436 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.637275 22440 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.639681 22435 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.631462 22444 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.640196 22435 ts_tablet_manager.cc:1403] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.007s
I20260709 12:11:47.641690 22444 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.642730 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:47.643658 22435 raft_consensus.cc:359] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
W20260709 12:11:47.644176 22208 leader_election.cc:343] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.644412 22435 raft_consensus.cc:385] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.644701 22435 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.645051 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
W20260709 12:11:47.646399 22357 leader_election.cc:343] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.646998 22440 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.647671 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:47.648667 22242 raft_consensus.cc:2393] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fc3e67a5db6c447583f943602f281536 in current term 1: Already voted for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in this term.
I20260709 12:11:47.646914 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:47.653883 22354 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:47.653272 22390 raft_consensus.cc:2393] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a6cb411c8bd34b8ca71f2df2d45e0458 in current term 1: Already voted for candidate fc3e67a5db6c447583f943602f281536 in this term.
I20260709 12:11:47.655464 22440 raft_consensus.cc:2749] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.656180 22206 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458; no voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536
I20260709 12:11:47.656970 22444 raft_consensus.cc:2749] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.656591 22435 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.659389 22435 ts_tablet_manager.cc:1434] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.004s
I20260709 12:11:47.660566 22435 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.668883 22435 tablet_bootstrap.cc:654] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.678875 22436 ts_tablet_manager.cc:1434] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.007s
I20260709 12:11:47.680058 22436 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.691093 22434 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.691542 22435 tablet_bootstrap.cc:492] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.692018 22435 ts_tablet_manager.cc:1403] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 12:11:47.691722 22434 ts_tablet_manager.cc:1403] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.016s
I20260709 12:11:47.695086 22435 raft_consensus.cc:359] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.695889 22435 raft_consensus.cc:385] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.696231 22435 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.696805 22456 raft_consensus.cc:493] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.697135 22435 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.697317 22456 raft_consensus.cc:515] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.695104 22436 tablet_bootstrap.cc:654] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.698864 22434 raft_consensus.cc:359] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.699692 22434 raft_consensus.cc:385] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.700063 22434 raft_consensus.cc:740] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.701123 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.700981 22434 consensus_queue.cc:260] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.702209 22456 leader_election.cc:290] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.702538 22435 ts_tablet_manager.cc:1434] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
W20260709 12:11:47.703233 22281 leader_election.cc:343] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.703544 22435 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.704120 22434 ts_tablet_manager.cc:1434] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:11:47.704444 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.705331 22434 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
W20260709 12:11:47.705811 22280 leader_election.cc:343] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.706236 22280 leader_election.cc:304] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536
I20260709 12:11:47.709178 22456 raft_consensus.cc:2749] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.712560 22435 tablet_bootstrap.cc:654] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.712985 22434 tablet_bootstrap.cc:654] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.716566 22440 raft_consensus.cc:493] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.717083 22440 raft_consensus.cc:515] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.720235 22440 leader_election.cc:290] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.721112 22436 tablet_bootstrap.cc:492] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.721679 22436 ts_tablet_manager.cc:1403] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.005s
I20260709 12:11:47.721840 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:47.723249 22317 raft_consensus.cc:2468] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 0.
I20260709 12:11:47.724429 22440 raft_consensus.cc:493] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.725003 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.724885 22440 raft_consensus.cc:515] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.725631 22436 raft_consensus.cc:359] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.726341 22436 raft_consensus.cc:385] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.726212 22357 leader_election.cc:304] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:47.726644 22436 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.726977 22440 leader_election.cc:290] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.727501 22440 raft_consensus.cc:2804] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.727969 22440 raft_consensus.cc:493] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.727442 22436 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.728495 22241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.729689 22456 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.730252 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
W20260709 12:11:47.735615 22354 leader_election.cc:343] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:47.736976 22357 leader_election.cc:343] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.736876 22456 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.739471 22456 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.739603 22436 ts_tablet_manager.cc:1434] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.005s
I20260709 12:11:47.733600 22440 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.740660 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.741413 22390 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 0.
I20260709 12:11:47.741706 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.742409 22242 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 0.
I20260709 12:11:47.742795 22281 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:47.743650 22456 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.744011 22456 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.744346 22456 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.745795 22472 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.746474 22472 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.750351 22472 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.750653 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
W20260709 12:11:47.751998 22208 leader_election.cc:343] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:11:47.753656 22354 leader_election.cc:343] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.753744 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.754145 22354 leader_election.cc:304] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:47.755247 22442 raft_consensus.cc:2749] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:11:47.755703 22206 leader_election.cc:343] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.756263 22206 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458; no voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536
I20260709 12:11:47.746773 22440 raft_consensus.cc:515] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.759063 22440 leader_election.cc:290] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.759536 22472 raft_consensus.cc:2749] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.760118 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:47.760665 22317 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.740641 22436 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.762295 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
W20260709 12:11:47.763588 22354 leader_election.cc:343] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:11:47.766952 22317 raft_consensus.cc:2468] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 1.
I20260709 12:11:47.768136 22357 leader_election.cc:304] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:47.768900 22440 raft_consensus.cc:2804] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:47.769367 22440 raft_consensus.cc:697] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:47.769229 22434 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.770076 22436 tablet_bootstrap.cc:654] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.770520 22434 ts_tablet_manager.cc:1403] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.005s
I20260709 12:11:47.770221 22440 consensus_queue.cc:237] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.774793 22456 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.779951 22435 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.780663 22435 ts_tablet_manager.cc:1403] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.012s
I20260709 12:11:47.784205 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:47.784811 22242 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.787109 22435 raft_consensus.cc:359] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.787905 22435 raft_consensus.cc:385] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.788237 22436 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.788272 22435 raft_consensus.cc:740] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.788825 22436 ts_tablet_manager.cc:1403] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.011s
I20260709 12:11:47.789335 22435 consensus_queue.cc:260] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.792097 22242 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:47.792656 22435 ts_tablet_manager.cc:1434] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:11:47.792459 22139 catalog_manager.cc:5697] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:47.793730 22435 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.793318 22280 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:47.796854 22441 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:47.797348 22441 raft_consensus.cc:697] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:47.798215 22441 consensus_queue.cc:237] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.804749 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:47.805379 22390 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.779209 22434 raft_consensus.cc:359] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.811180 22434 raft_consensus.cc:385] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.811527 22434 raft_consensus.cc:740] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.812587 22390 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:47.812319 22434 consensus_queue.cc:260] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.814733 22434 ts_tablet_manager.cc:1434] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.004s
I20260709 12:11:47.815764 22434 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.795650 22436 raft_consensus.cc:359] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.816381 22137 catalog_manager.cc:5697] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:47.817610 22436 raft_consensus.cc:385] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.817952 22436 raft_consensus.cc:740] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.818907 22436 consensus_queue.cc:260] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.818005 22456 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.821498 22435 tablet_bootstrap.cc:654] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.823498 22436 ts_tablet_manager.cc:1434] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.002s
I20260709 12:11:47.833827 22435 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.835070 22435 ts_tablet_manager.cc:1403] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.007s
I20260709 12:11:47.835374 22436 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.838461 22435 raft_consensus.cc:359] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.839313 22435 raft_consensus.cc:385] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.839608 22435 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.840562 22435 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.842949 22436 tablet_bootstrap.cc:654] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.843242 22435 ts_tablet_manager.cc:1434] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:11:47.844316 22435 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: Bootstrap starting.
I20260709 12:11:47.850451 22435 tablet_bootstrap.cc:654] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.855741 22436 tablet_bootstrap.cc:492] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.856341 22436 ts_tablet_manager.cc:1403] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.010s
I20260709 12:11:47.859360 22436 raft_consensus.cc:359] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.860280 22436 raft_consensus.cc:385] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.860685 22436 raft_consensus.cc:740] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.861575 22436 consensus_queue.cc:260] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.864120 22434 tablet_bootstrap.cc:654] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.864149 22436 ts_tablet_manager.cc:1434] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:11:47.869349 22436 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.884023 22435 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: No bootstrap required, opened a new log
I20260709 12:11:47.884588 22435 ts_tablet_manager.cc:1403] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.017s
I20260709 12:11:47.887491 22435 raft_consensus.cc:359] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.891059 22435 raft_consensus.cc:385] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.891455 22435 raft_consensus.cc:740] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.891690 22436 tablet_bootstrap.cc:654] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.892405 22435 consensus_queue.cc:260] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.893941 22434 tablet_bootstrap.cc:492] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.894440 22434 ts_tablet_manager.cc:1403] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.000s
I20260709 12:11:47.900051 22435 ts_tablet_manager.cc:1434] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:11:47.902778 22434 raft_consensus.cc:359] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.903541 22434 raft_consensus.cc:385] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.903873 22434 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.904716 22434 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.906162 22456 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:47.906720 22434 ts_tablet_manager.cc:1434] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:11:47.908978 22434 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: Bootstrap starting.
I20260709 12:11:47.914554 22434 tablet_bootstrap.cc:654] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.923580 22436 tablet_bootstrap.cc:492] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:47.924177 22436 ts_tablet_manager.cc:1403] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.014s
I20260709 12:11:47.927752 22436 raft_consensus.cc:359] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.928530 22436 raft_consensus.cc:385] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.928802 22434 tablet_bootstrap.cc:492] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: No bootstrap required, opened a new log
I20260709 12:11:47.928853 22436 raft_consensus.cc:740] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.929354 22434 ts_tablet_manager.cc:1403] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 12:11:47.929811 22436 consensus_queue.cc:260] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.931275 22456 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:47.931939 22436 ts_tablet_manager.cc:1434] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:11:47.932969 22436 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: Bootstrap starting.
I20260709 12:11:47.932924 22472 raft_consensus.cc:493] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.933651 22472 raft_consensus.cc:515] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.934023 22434 raft_consensus.cc:359] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.934724 22434 raft_consensus.cc:385] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:47.935020 22434 raft_consensus.cc:740] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Initialized, Role: FOLLOWER
I20260709 12:11:47.935860 22434 consensus_queue.cc:260] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.936251 22472 leader_election.cc:290] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:47.937860 22434 ts_tablet_manager.cc:1434] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:11:47.937809 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:47.938827 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:47.939603 22317 raft_consensus.cc:2393] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6cb411c8bd34b8ca71f2df2d45e0458 in current term 1: Already voted for candidate fc3e67a5db6c447583f943602f281536 in this term.
I20260709 12:11:47.940770 22208 leader_election.cc:304] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458; no voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536
I20260709 12:11:47.945112 22436 tablet_bootstrap.cc:654] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: Neither blocks nor log segments found. Creating new log.
I20260709 12:11:47.948094 22444 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.954659 22440 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.955173 22440 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.957026 22440 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.957887 22241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:47.958549 22241 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 0.
I20260709 12:11:47.959873 22354 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:47.960798 22440 raft_consensus.cc:2804] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:47.961283 22440 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.961705 22440 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.962253 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:47.962896 22317 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 0.
I20260709 12:11:47.954123 22444 raft_consensus.cc:2749] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:47.970507 22440 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:47.974577 22241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:47.975394 22241 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.986562 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:47.986943 22241 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 1.
I20260709 12:11:47.987282 22317 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:47.987282 22440 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:47.997874 22354 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:47.999035 22440 raft_consensus.cc:2804] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:47.999547 22440 raft_consensus.cc:697] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:48.000393 22440 consensus_queue.cc:237] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.001080 22317 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 1.
I20260709 12:11:48.009503 22436 tablet_bootstrap.cc:492] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: No bootstrap required, opened a new log
I20260709 12:11:48.009994 22436 ts_tablet_manager.cc:1403] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent bootstrapping tablet: real 0.077s	user 0.022s	sys 0.020s
I20260709 12:11:48.012727 22436 raft_consensus.cc:359] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.012605 22137 catalog_manager.cc:5697] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:48.013460 22436 raft_consensus.cc:385] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:11:48.013849 22436 raft_consensus.cc:740] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Initialized, Role: FOLLOWER
I20260709 12:11:48.014672 22436 consensus_queue.cc:260] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.017304 22472 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.017925 22493 raft_consensus.cc:493] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.017823 22472 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:48.018400 22493 raft_consensus.cc:515] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.019980 22472 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:48.020753 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:48.021044 22441 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.021528 22441 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.023725 22441 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.024133 22436 ts_tablet_manager.cc:1434] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:11:48.024461 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:48.025249 22317 raft_consensus.cc:2393] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6cb411c8bd34b8ca71f2df2d45e0458 in current term 1: Already voted for candidate fc3e67a5db6c447583f943602f281536 in this term.
I20260709 12:11:48.026372 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:48.027125 22391 raft_consensus.cc:2393] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd4d6e7b1323429c8880fe673141ca39 in current term 1: Already voted for candidate fc3e67a5db6c447583f943602f281536 in this term.
I20260709 12:11:48.026942 22208 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458; no voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536
I20260709 12:11:48.028636 22472 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.028519 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:48.029496 22242 raft_consensus.cc:2393] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd4d6e7b1323429c8880fe673141ca39 in current term 1: Already voted for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in this term.
I20260709 12:11:48.030599 22280 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536
I20260709 12:11:48.032713 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:48.033331 22317 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.034235 22444 raft_consensus.cc:493] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.034507 22441 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.034673 22208 leader_election.cc:304] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:48.034782 22444 raft_consensus.cc:515] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.035666 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:48.036324 22391 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.037317 22444 leader_election.cc:290] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.038045 22444 raft_consensus.cc:2804] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:48.038409 22444 raft_consensus.cc:493] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.038748 22444 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.038697 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:48.039536 22391 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.040766 22206 leader_election.cc:304] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:48.043545 22472 raft_consensus.cc:2749] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:48.041638 22441 raft_consensus.cc:2749] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:11:48.045104 22472 raft_consensus.cc:2804] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:11:48.045288 22187 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:48.045710 22472 raft_consensus.cc:493] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.041927 22493 leader_election.cc:290] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.046319 22472 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.052769 22444 raft_consensus.cc:515] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.053695 22472 raft_consensus.cc:515] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.055809 22472 leader_election.cc:290] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.055737 22444 leader_election.cc:290] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.057278 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:48.057674 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:48.057967 22391 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.058688 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:48.059333 22317 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.058295 22390 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.060515 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:48.061666 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:48.062112 22316 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.062420 22315 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.068806 22390 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.070211 22206 leader_election.cc:304] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:48.072928 22315 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.073690 22316 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.074209 22391 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.075016 22444 raft_consensus.cc:2804] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:48.074921 22208 leader_election.cc:304] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:48.076107 22494 raft_consensus.cc:2804] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:48.076581 22494 raft_consensus.cc:697] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:48.077265 22494 consensus_queue.cc:237] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.080621 22444 raft_consensus.cc:697] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:48.081878 22444 consensus_queue.cc:237] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.089803 22440 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.092798 22136 catalog_manager.cc:5697] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:48.098948 22440 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.115911 22136 catalog_manager.cc:5697] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:48.260167 22444 raft_consensus.cc:493] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.260668 22444 raft_consensus.cc:515] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.262516 22444 leader_election.cc:290] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.263285 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:11:48.263677 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:48.263971 22316 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.264264 22391 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 0.
I20260709 12:11:48.265203 22208 leader_election.cc:304] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:48.265923 22444 raft_consensus.cc:2804] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:11:48.266292 22444 raft_consensus.cc:493] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.266638 22444 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.273413 22444 raft_consensus.cc:515] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.275517 22444 leader_election.cc:290] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:48.276459 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:48.276827 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:48.277014 22316 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.277400 22391 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:11:48.282747 22316 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.283815 22391 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 1.
I20260709 12:11:48.284458 22208 leader_election.cc:304] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:48.285230 22444 raft_consensus.cc:2804] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:11:48.285706 22444 raft_consensus.cc:697] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:48.286499 22444 consensus_queue.cc:237] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:48.294672 22139 catalog_manager.cc:5697] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:48.304759 22441 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.315819 22456 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.335295 22498 consensus_queue.cc:1048] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.348049 22440 consensus_queue.cc:1048] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.462958 22440 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.473255 22498 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.495337 22444 consensus_queue.cc:1048] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.507110 22493 consensus_queue.cc:1048] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.628937 22493 consensus_queue.cc:1048] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.640064 22444 consensus_queue.cc:1048] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.797013 22444 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:48.810447 22444 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:11:49.047835 22187 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:11:49.650305 22441 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.650823 22441 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:49.652516 22441 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:49.653352 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" is_pre_election: true
I20260709 12:11:49.653517 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:11:49.653944 22242 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:49.654037 22391 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 1.
I20260709 12:11:49.655045 22281 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:49.655694 22441 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:11:49.656061 22441 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.656302 22441 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:49.661085 22441 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:49.662624 22441 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:49.663224 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:49.663503 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:49.663717 22242 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:49.664031 22391 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:49.669104 22242 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:49.669334 22391 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:49.670094 22280 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:49.670713 22441 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:49.671114 22441 raft_consensus.cc:697] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:49.671718 22441 consensus_queue.cc:237] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:49.679082 22137 catalog_manager.cc:5697] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 0 to 2, leader changed from <none> to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.086632 22242 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:11:50.087963 22441 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:50.097570 22391 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:11:50.099397 22521 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:11:50.302096 22391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:33630
I20260709 12:11:50.302690 22391 raft_consensus.cc:606] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:50.315960 22315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58720
I20260709 12:11:50.316502 22315 raft_consensus.cc:606] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:50.322007 22315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58720
I20260709 12:11:50.322551 22315 raft_consensus.cc:606] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:50.327616 22315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58720
I20260709 12:11:50.328094 22315 raft_consensus.cc:606] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:11:50.333135 22391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:33630
I20260709 12:11:50.333719 22391 raft_consensus.cc:606] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:50.347188 22242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:11:50.347821 22242 raft_consensus.cc:606] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:50.353034 22242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3468b676b534142851875be523c6ee0"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:11:50.353535 22242 raft_consensus.cc:606] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:50.358436 22391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:33630
I20260709 12:11:50.359097 22441 raft_consensus.cc:993] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.359007 22391 raft_consensus.cc:606] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:11:50.359598 22521 raft_consensus.cc:1081] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.361516 22242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:11:50.362003 22242 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.362303 22242 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.364348 22241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:11:50.364874 22241 raft_consensus.cc:606] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Received request to transfer leadership to TS fc3e67a5db6c447583f943602f281536
I20260709 12:11:50.367612 22242 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.369419 22242 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.370177 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:50.370455 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.370723 22315 raft_consensus.cc:3055] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.370973 22391 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.371094 22315 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.371897 22315 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.372913 22315 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.375721 22391 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.376755 22206 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:50.377475 22444 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.378705 22444 raft_consensus.cc:697] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.379500 22315 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.379453 22444 consensus_queue.cc:237] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.386914 22531 raft_consensus.cc:993] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.387390 22440 raft_consensus.cc:1081] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.388451 22137 catalog_manager.cc:5697] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 1 to 2, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.389117 22315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47396
I20260709 12:11:50.389904 22315 raft_consensus.cc:493] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.390230 22315 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.396334 22315 raft_consensus.cc:515] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.399129 22315 leader_election.cc:290] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.399314 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:50.399523 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59269a8cb6ec4500b34f4e7292005694" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.400028 22391 raft_consensus.cc:3055] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.399976 22242 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.400506 22391 raft_consensus.cc:740] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:50.401340 22391 consensus_queue.cc:260] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.402606 22391 raft_consensus.cc:3060] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.407187 22242 raft_consensus.cc:2468] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.408496 22280 leader_election.cc:304] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:50.409235 22521 raft_consensus.cc:2804] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.409286 22391 raft_consensus.cc:2468] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.409852 22521 raft_consensus.cc:697] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.410688 22521 consensus_queue.cc:237] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.420291 22137 catalog_manager.cc:5697] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 1 to 2, leader changed from fc3e67a5db6c447583f943602f281536 (127.21.150.195) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.497534 22529 raft_consensus.cc:993] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.497980 22444 raft_consensus.cc:1081] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.499429 22315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:11:50.499929 22315 raft_consensus.cc:493] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.500228 22315 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.505007 22315 raft_consensus.cc:515] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.506639 22315 leader_election.cc:290] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.507501 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:50.507670 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03c8c69d2f8c424799063791f4f92c1d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.508078 22242 raft_consensus.cc:3055] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.508270 22391 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.508498 22242 raft_consensus.cc:740] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.509244 22242 consensus_queue.cc:260] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.510457 22242 raft_consensus.cc:3060] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.513242 22391 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.514333 22281 leader_election.cc:304] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:50.515229 22521 raft_consensus.cc:2804] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.516005 22521 raft_consensus.cc:697] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.515441 22242 raft_consensus.cc:2468] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.516881 22521 consensus_queue.cc:237] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.522202 22531 raft_consensus.cc:993] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.522801 22440 raft_consensus.cc:1081] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.524366 22315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47396
I20260709 12:11:50.524863 22315 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.525170 22315 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.525058 22137 catalog_manager.cc:5697] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 1 to 2, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.531348 22315 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.534158 22315 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.534818 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.534868 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:50.535543 22391 raft_consensus.cc:3055] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.535617 22242 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.535848 22391 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:50.536625 22391 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.537593 22391 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.541877 22242 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.542476 22391 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.542930 22280 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:50.543711 22521 raft_consensus.cc:2804] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.544149 22521 raft_consensus.cc:697] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.544847 22521 consensus_queue.cc:237] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.551506 22137 catalog_manager.cc:5697] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 1 to 2, leader changed from fc3e67a5db6c447583f943602f281536 (127.21.150.195) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.632124 22441 raft_consensus.cc:993] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.632572 22521 raft_consensus.cc:1081] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.633936 22242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:11:50.634403 22242 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.634709 22242 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:50.639029 22242 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.640583 22242 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 3 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.641355 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:50.641602 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.641966 22315 raft_consensus.cc:3055] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:50.642119 22391 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:50.642328 22315 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.643149 22315 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.644341 22315 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:50.648636 22391 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 3.
I20260709 12:11:50.649712 22206 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:50.650386 22444 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:50.650471 22315 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 3.
I20260709 12:11:50.650908 22444 raft_consensus.cc:697] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.651710 22444 consensus_queue.cc:237] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.658138 22137 catalog_manager.cc:5697] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 2 to 3, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.702278 22531 raft_consensus.cc:993] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.702841 22440 raft_consensus.cc:1081] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.704463 22242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37330
I20260709 12:11:50.704937 22242 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.705216 22242 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.710352 22242 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:50.712199 22242 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Requested vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:50.713107 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.713191 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:50.713907 22315 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.713907 22391 raft_consensus.cc:3055] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.714447 22391 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:50.715327 22391 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:50.716550 22391 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.721215 22315 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.722697 22208 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:50.723421 22391 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.723585 22444 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.724189 22444 raft_consensus.cc:697] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.724931 22444 consensus_queue.cc:237] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:11:50.731685 22137 catalog_manager.cc:5697] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 1 to 2, leader changed from fc3e67a5db6c447583f943602f281536 (127.21.150.195) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.759981 22529 raft_consensus.cc:993] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:50.760416 22444 raft_consensus.cc:1081] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Signalling peer fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:50.761777 22391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:38638
I20260709 12:11:50.762238 22391 raft_consensus.cc:493] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.762527 22391 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.767041 22391 raft_consensus.cc:515] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.769120 22391 leader_election.cc:290] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:50.769380 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:50.769721 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af9f742961e44504ab2db17c60a2730b" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:50.770032 22242 raft_consensus.cc:3055] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.770243 22315 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.770372 22242 raft_consensus.cc:740] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.771205 22242 consensus_queue.cc:260] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.772279 22242 raft_consensus.cc:3060] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.776501 22315 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 2.
I20260709 12:11:50.777549 22357 leader_election.cc:304] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:50.777930 22242 raft_consensus.cc:2468] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 2.
I20260709 12:11:50.778213 22440 raft_consensus.cc:2804] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.778678 22440 raft_consensus.cc:697] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 2 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:50.779474 22440 consensus_queue.cc:237] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.785904 22137 catalog_manager.cc:5697] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 1 to 2, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:50.794499 22529 raft_consensus.cc:993] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.794880 22444 raft_consensus.cc:1081] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:11:50.796175 22315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3468b676b534142851875be523c6ee0"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:11:50.796649 22315 raft_consensus.cc:493] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.796952 22315 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.801280 22315 raft_consensus.cc:515] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.803207 22315 leader_election.cc:290] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.803874 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:50.804216 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.804399 22242 raft_consensus.cc:3055] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.804711 22242 raft_consensus.cc:740] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.804747 22391 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.805315 22242 consensus_queue.cc:260] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.806135 22242 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.810155 22391 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.810173 22242 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 2.
I20260709 12:11:50.811228 22280 leader_election.cc:304] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:11:50.811961 22521 raft_consensus.cc:2804] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.812379 22521 raft_consensus.cc:697] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.813143 22521 consensus_queue.cc:237] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.820638 22137 catalog_manager.cc:5697] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 1 to 2, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.829282 22315 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.831079 22444 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.836777 22441 raft_consensus.cc:993] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.837222 22521 raft_consensus.cc:1081] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:11:50.841414 22391 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.841372 22242 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:11:50.842787 22242 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:50.843205 22444 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.843935 22242 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.855072 22242 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.856804 22242 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:11:50.857594 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:50.858255 22315 raft_consensus.cc:3055] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:11:50.858573 22315 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:50.858503 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:11:50.859148 22391 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.859478 22315 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.860638 22315 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:11:50.865413 22391 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.866437 22206 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:50.866668 22315 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 2.
I20260709 12:11:50.867139 22444 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:11:50.867609 22444 raft_consensus.cc:697] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:50.868381 22444 consensus_queue.cc:237] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:50.874742 22137 catalog_manager.cc:5697] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 1 to 2, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.894693 22391 raft_consensus.cc:1275] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.895768 22521 consensus_queue.cc:1048] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.905599 22242 raft_consensus.cc:1275] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.907250 22549 consensus_queue.cc:1048] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.954968 22391 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.956089 22549 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.964247 22242 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.965799 22521 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.969748 22241 raft_consensus.cc:1275] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.971397 22521 consensus_queue.cc:1048] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:50.980329 22391 raft_consensus.cc:1275] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:50.982573 22554 consensus_queue.cc:1048] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:11:51.049017 22391 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:11:51.050213 22444 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.056980 22315 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:11:51.058566 22543 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.169032 22242 raft_consensus.cc:1275] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.170341 22440 consensus_queue.cc:1048] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.179092 22315 raft_consensus.cc:1275] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.181071 22557 consensus_queue.cc:1048] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.245385 22315 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.246690 22543 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.255119 22391 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.256695 22444 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.288574 22391 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.289772 22543 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.297822 22315 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.299350 22444 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.390619 22242 raft_consensus.cc:1275] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.391916 22549 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:51.404919 22391 raft_consensus.cc:1275] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:11:51.406703 22521 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:11:53.397001 22107 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:11:53.398264 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:53.421633 22315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:11:53.422194 22315 raft_consensus.cc:606] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Received request to transfer leadership to TS fc3e67a5db6c447583f943602f281536
I20260709 12:11:53.433053 22242 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:48032
I20260709 12:11:53.433564 22242 raft_consensus.cc:606] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Received request to transfer leadership to TS fc3e67a5db6c447583f943602f281536
I20260709 12:11:53.434624 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:11:53.435019 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:53.454767 22521 raft_consensus.cc:993] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:53.455199 22568 raft_consensus.cc:1081] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Signalling peer fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:53.456617 22390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:38626
I20260709 12:11:53.457064 22390 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:53.457317 22390 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.461624 22390 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.463431 22390 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:53.464177 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:53.464516 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:53.464700 22242 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.465071 22315 raft_consensus.cc:3055] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:53.465374 22315 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:11:53.466106 22315 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.466943 22315 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.468812 22242 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:11:53.469774 22354 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:53.470448 22440 raft_consensus.cc:2804] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:53.471621 22440 raft_consensus.cc:697] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:53.471884 22315 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:11:53.472479 22440 consensus_queue.cc:237] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.479552 22137 catalog_manager.cc:5697] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 2 to 3, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.743464 22576 raft_consensus.cc:993] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:53.743916 22576 raft_consensus.cc:1081] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Signalling peer fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:11:53.745711 22391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:38638
I20260709 12:11:53.746246 22391 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:53.746536 22391 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.750592 22391 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.751945 22391 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:11:53.752948 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:11:53.753077 22315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:11:53.753563 22242 raft_consensus.cc:3055] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:11:53.753620 22315 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.753881 22242 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:11:53.754580 22242 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.755405 22242 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:11:53.757993 22315 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:11:53.758998 22357 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:11:53.759754 22440 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:11:53.760167 22440 raft_consensus.cc:697] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:11:53.760212 22242 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:11:53.761094 22440 consensus_queue.cc:237] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:11:53.767689 22137 catalog_manager.cc:5697] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 2 to 3, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:53.876434 22315 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:53.877915 22440 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:53.889007 22242 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:53.890293 22440 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:54.151711 22315 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:54.153015 22440 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:54.162891 22242 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:11:54.164359 22440 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:11:54.435941 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:54.442417 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:54.442854 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:55.443977 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:55.448985 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:55.449362 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:56.450553 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:56.455905 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:56.456264 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:57.457284 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:57.462158 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:57.462538 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:58.463413 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:58.468226 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:58.468641 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:59.469640 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:59.474396 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:11:59.474753 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:00.475625 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:00.481372 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:00.481745 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:01.482915 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:01.488842 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:01.489256 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:02.490291 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:02.494967 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:02.495329 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:03.522639 22107 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:12:03.527302 22241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:12:03.527976 22241 raft_consensus.cc:606] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:03.533141 22241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:12:03.533649 22241 raft_consensus.cc:606] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:03.538676 22390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:33630
I20260709 12:12:03.539168 22390 raft_consensus.cc:606] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:03.543809 22241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:48024
I20260709 12:12:03.544242 22241 raft_consensus.cc:606] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:03.556206 22390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
mode: GRACEFUL
new_leader_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:33630
I20260709 12:12:03.556806 22390 raft_consensus.cc:606] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Received request to transfer leadership to TS cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:03.622040 22592 raft_consensus.cc:993] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.622529 22592 raft_consensus.cc:1081] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.624070 22316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:12:03.624557 22316 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:03.624887 22316 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:03.630054 22316 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.632437 22316 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 3 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:03.633666 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:03.633730 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:03.634598 22390 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:03.634670 22242 raft_consensus.cc:3055] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:03.635007 22242 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:03.635670 22242 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.636761 22242 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:03.639685 22390 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 3.
I20260709 12:12:03.640880 22281 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:03.641136 22242 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 3.
I20260709 12:12:03.641848 22590 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:03.642765 22590 raft_consensus.cc:697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:03.643576 22590 consensus_queue.cc:237] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.650523 22137 catalog_manager.cc:5697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 2 to 3, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:03.653563 22595 raft_consensus.cc:993] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.654618 22587 raft_consensus.cc:1081] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.656625 22315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47396
I20260709 12:12:03.657124 22315 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:03.657505 22315 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.658313 22595 raft_consensus.cc:993] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.658936 22597 raft_consensus.cc:1081] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.660454 22316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47396
I20260709 12:12:03.660900 22316 raft_consensus.cc:493] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:03.661207 22316 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.665338 22316 raft_consensus.cc:515] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.665958 22315 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.667847 22315 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:03.669103 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:03.669510 22241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:03.669693 22242 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.670079 22241 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.671543 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "148682670bd0400bbe7e8e3545f7937d" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:03.672043 22391 raft_consensus.cc:3055] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:03.672322 22391 raft_consensus.cc:740] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:12:03.673079 22391 consensus_queue.cc:260] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.674047 22316 leader_election.cc:290] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:03.674131 22391 raft_consensus.cc:3060] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.669700 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:03.676132 22390 raft_consensus.cc:3055] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:03.676429 22390 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:12:03.677135 22241 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.677192 22390 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.678521 22390 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.678634 22280 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:12:03.679469 22590 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:03.679912 22590 raft_consensus.cc:697] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:03.680480 22391 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.680713 22590 consensus_queue.cc:237] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.683833 22390 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.681285 22242 raft_consensus.cc:2468] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.681741 22281 leader_election.cc:304] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:03.685588 22593 raft_consensus.cc:2804] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:03.686786 22593 raft_consensus.cc:697] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:03.687553 22593 consensus_queue.cc:237] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.695374 22137 catalog_manager.cc:5697] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 3 to 4, leader changed from fc3e67a5db6c447583f943602f281536 (127.21.150.195) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:03.705811 22137 catalog_manager.cc:5697] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 3 to 4, leader changed from fc3e67a5db6c447583f943602f281536 (127.21.150.195) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:03.840088 22565 raft_consensus.cc:993] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.840521 22592 raft_consensus.cc:1081] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:03.842002 22316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:12:03.842497 22316 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:03.842813 22316 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.847255 22316 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.848843 22316 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:03.849689 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:03.849818 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:03.850262 22242 raft_consensus.cc:3055] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:03.850296 22390 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.850561 22242 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:03.851150 22242 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.852008 22242 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:03.854465 22390 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.855468 22281 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:03.856323 22593 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:03.856566 22242 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:03.856766 22593 raft_consensus.cc:697] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:03.857815 22593 consensus_queue.cc:237] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:03.866871 22137 catalog_manager.cc:5697] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 3 to 4, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:04.044047 22242 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:04.045423 22593 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:04.055526 22390 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:04.057142 22593 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:04.061211 22592 raft_consensus.cc:993] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: : Instructing follower cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:04.061717 22592 raft_consensus.cc:1081] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Signalling peer cd4d6e7b1323429c8880fe673141ca39 to start an election
I20260709 12:12:04.063534 22316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
 from {username='slave'} at 127.0.0.1:47408
I20260709 12:12:04.064080 22316 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:04.064505 22316 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:04.071225 22316 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:04.072968 22316 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 3 election: Requested vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:12:04.074290 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:04.073637 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:04.074826 22242 raft_consensus.cc:3055] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:04.074940 22390 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:04.075169 22242 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:04.075894 22241 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:04.075922 22242 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:04.077093 22242 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:04.078238 22590 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:04.081700 22391 raft_consensus.cc:1275] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:04.083277 22593 consensus_queue.cc:1048] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:04.084437 22242 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 3.
I20260709 12:12:04.085810 22280 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39; no voters: 
I20260709 12:12:04.086810 22590 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:04.087494 22590 raft_consensus.cc:697] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:04.090191 22390 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 3.
I20260709 12:12:04.088711 22590 consensus_queue.cc:237] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:04.101840 22136 catalog_manager.cc:5697] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 2 to 3, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:04.158761 22391 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:04.160125 22613 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:04.169651 22240 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:04.171044 22602 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:04.252056 22240 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:04.253552 22602 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:04.264858 22391 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:12:04.266052 22602 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:04.573695 22391 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:04.575328 22602 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:04.614478 22240 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Refusing update from remote peer cd4d6e7b1323429c8880fe673141ca39: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:04.620419 22593 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:06.590674 22107 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:12:06.591689 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:06.598668 22316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:12:06.599289 22316 raft_consensus.cc:606] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:12:06.602031 22316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:12:06.602555 22316 raft_consensus.cc:606] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Received request to transfer leadership to TS fc3e67a5db6c447583f943602f281536
I20260709 12:12:06.604820 22316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:12:06.605340 22316 raft_consensus.cc:606] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:12:06.607395 22316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3468b676b534142851875be523c6ee0"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:12:06.607833 22316 raft_consensus.cc:606] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Received request to transfer leadership to TS fc3e67a5db6c447583f943602f281536
I20260709 12:12:06.610150 22316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
mode: GRACEFUL
new_leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:58722
I20260709 12:12:06.610615 22316 raft_consensus.cc:606] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Received request to transfer leadership to TS a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:12:06.611630 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:06.612113 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:06.768153 22625 raft_consensus.cc:993] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:12:06.768677 22593 raft_consensus.cc:1081] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Signalling peer fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:12:06.770430 22391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:38626
I20260709 12:12:06.770947 22391 raft_consensus.cc:493] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:06.771253 22391 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.776234 22391 raft_consensus.cc:515] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.778167 22391 leader_election.cc:290] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:12:06.779342 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:06.779335 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a57670b155f42dabe1699c849e90e92" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:12:06.780159 22242 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.780251 22316 raft_consensus.cc:3055] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:06.780678 22316 raft_consensus.cc:740] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:06.781529 22316 consensus_queue.cc:260] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.782565 22316 raft_consensus.cc:3060] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.784895 22242 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 5.
I20260709 12:12:06.786149 22354 leader_election.cc:304] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:06.787088 22622 raft_consensus.cc:2804] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:06.788048 22622 raft_consensus.cc:697] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 5 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:12:06.788229 22316 raft_consensus.cc:2468] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 5.
I20260709 12:12:06.789055 22622 consensus_queue.cc:237] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.798705 22136 catalog_manager.cc:5697] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 4 to 5, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 5 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:06.841493 22625 raft_consensus.cc:993] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:12:06.841933 22593 raft_consensus.cc:1081] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Signalling peer fc3e67a5db6c447583f943602f281536 to start an election
I20260709 12:12:06.843282 22391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3468b676b534142851875be523c6ee0"
dest_uuid: "fc3e67a5db6c447583f943602f281536"
 from {username='slave'} at 127.0.0.1:38626
I20260709 12:12:06.843758 22391 raft_consensus.cc:493] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:06.844043 22391 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:06.848738 22391 raft_consensus.cc:515] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.850137 22391 leader_election.cc:290] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:12:06.851042 22242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
I20260709 12:12:06.851126 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3468b676b534142851875be523c6ee0" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:12:06.851513 22242 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:06.851737 22316 raft_consensus.cc:3055] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:06.852103 22316 raft_consensus.cc:740] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:06.852878 22316 consensus_queue.cc:260] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.853936 22316 raft_consensus.cc:3060] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:06.855703 22242 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:12:06.856731 22354 leader_election.cc:304] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:06.857491 22622 raft_consensus.cc:2804] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:06.858057 22622 raft_consensus.cc:697] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Becoming Leader. State: Replica: fc3e67a5db6c447583f943602f281536, State: Running, Role: LEADER
I20260709 12:12:06.859030 22622 consensus_queue.cc:237] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.860198 22316 raft_consensus.cc:2468] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 3.
I20260709 12:12:06.866818 22136 catalog_manager.cc:5697] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 reported cstate change: term changed from 2 to 3, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to fc3e67a5db6c447583f943602f281536 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fc3e67a5db6c447583f943602f281536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:06.920959 22625 raft_consensus.cc:993] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:06.921391 22593 raft_consensus.cc:1081] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:06.923183 22240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:12:06.923725 22240 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:06.924072 22240 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.931035 22240 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.934139 22240 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 5 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:06.935700 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:12:06.935966 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:06.936265 22316 raft_consensus.cc:3055] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:12:06.936496 22391 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.936635 22316 raft_consensus.cc:740] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:06.937565 22316 consensus_queue.cc:260] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.938446 22316 raft_consensus.cc:3060] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:06.941028 22391 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 5.
I20260709 12:12:06.942588 22206 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:06.944064 22316 raft_consensus.cc:2468] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 5.
I20260709 12:12:06.944360 22633 raft_consensus.cc:2804] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:06.945973 22633 raft_consensus.cc:697] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:06.946970 22633 consensus_queue.cc:237] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:06.955479 22136 catalog_manager.cc:5697] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 4 to 5, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 5 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:07.014600 22625 raft_consensus.cc:993] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:07.015125 22593 raft_consensus.cc:1081] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:07.016732 22240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:12:07.017303 22240 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:07.017681 22240 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.022998 22240 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:07.024767 22240 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 4 election: Requested vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:12:07.025810 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:07.025986 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:12:07.026412 22391 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.026491 22316 raft_consensus.cc:3055] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:07.026938 22316 raft_consensus.cc:740] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:07.027745 22316 consensus_queue.cc:260] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:07.028990 22316 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.033218 22391 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 4.
I20260709 12:12:07.034482 22206 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:07.035257 22633 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:07.035763 22633 raft_consensus.cc:697] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:07.035308 22316 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 4.
I20260709 12:12:07.037225 22633 consensus_queue.cc:237] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:07.044273 22136 catalog_manager.cc:5697] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 3 to 4, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:07.200100 22625 raft_consensus.cc:993] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: : Instructing follower a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:07.200649 22625 raft_consensus.cc:1081] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Signalling peer a6cb411c8bd34b8ca71f2df2d45e0458 to start an election
I20260709 12:12:07.202872 22240 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc"
dest_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458"
 from {username='slave'} at 127.0.0.1:37340
I20260709 12:12:07.203408 22240 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:07.203683 22240 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.209518 22240 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:07.211719 22240 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 4 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:07.212419 22316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cd4d6e7b1323429c8880fe673141ca39"
I20260709 12:12:07.212751 22391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:07.212962 22316 raft_consensus.cc:3055] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:07.213250 22391 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.213286 22316 raft_consensus.cc:740] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:07.214184 22316 consensus_queue.cc:260] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:07.215315 22316 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:07.218077 22391 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 4.
I20260709 12:12:07.219161 22206 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:07.219630 22316 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6cb411c8bd34b8ca71f2df2d45e0458 in term 4.
I20260709 12:12:07.219952 22633 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:07.220709 22633 raft_consensus.cc:697] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 LEADER]: Becoming Leader. State: Replica: a6cb411c8bd34b8ca71f2df2d45e0458, State: Running, Role: LEADER
I20260709 12:12:07.221632 22633 consensus_queue.cc:237] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:07.231407 22136 catalog_manager.cc:5697] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 reported cstate change: term changed from 3 to 4, leader changed from cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194) to a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:07.252861 22240 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:07.254746 22622 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:12:07.262243 22316 raft_consensus.cc:1275] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:12:07.263307 22315 raft_consensus.cc:1275] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:07.264143 22626 consensus_queue.cc:1048] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:12:07.265776 22626 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:07.292595 22240 raft_consensus.cc:1275] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Refusing update from remote peer fc3e67a5db6c447583f943602f281536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:07.294770 22639 consensus_queue.cc:1048] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:12:07.438457 22316 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:07.439668 22633 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:07.448912 22391 raft_consensus.cc:1275] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:07.450708 22634 consensus_queue.cc:1048] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:07.473208 22316 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:12:07.474488 22634 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:07.483564 22391 raft_consensus.cc:1275] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:12:07.485859 22633 consensus_queue.cc:1048] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:07.613538 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:07.618316 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:07.618760 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:07.675257 22316 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:07.675432 22390 raft_consensus.cc:1275] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Refusing update from remote peer a6cb411c8bd34b8ca71f2df2d45e0458: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:07.676994 22652 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:07.677711 22633 consensus_queue.cc:1048] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:08.619943 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:08.625715 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:08.626190 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:09.627372 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:09.633289 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:09.634742 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:10.635984 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:10.641762 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:10.642304 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:11.643515 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:11.648999 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:11.649521 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:12.650624 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:12.655648 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:12.656106 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:13.657702 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:13.663961 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:13.664420 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:14.666142 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:14.674058 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:14.674600 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:12:15.808945 22263 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7060 after lost signal to thread 22110
W20260709 12:12:15.809880 22263 debug-util.cc:398] Leaking SignalData structure 0x7b08000e7740 after lost signal to thread 22173
W20260709 12:12:15.810792 22263 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2cc0 after lost signal to thread 22266
W20260709 12:12:15.811700 22263 debug-util.cc:398] Leaking SignalData structure 0x7b08001da420 after lost signal to thread 22341
W20260709 12:12:15.812392 22263 debug-util.cc:398] Leaking SignalData structure 0x7b080010dc80 after lost signal to thread 22415
I20260709 12:12:15.982314 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:15.987290 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:15.987720 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:17.027913 22107 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:12:17.028736 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
I20260709 12:12:17.090443 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:17.091967 22107 tablet_replica.cc:333] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.093890 22107 raft_consensus.cc:2243] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.095211 22107 raft_consensus.cc:2272] T 59269a8cb6ec4500b34f4e7292005694 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.098390 22107 tablet_replica.cc:333] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.099006 22107 raft_consensus.cc:2243] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.099807 22107 raft_consensus.cc:2272] T af9f742961e44504ab2db17c60a2730b P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.102133 22107 tablet_replica.cc:333] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.102741 22107 raft_consensus.cc:2243] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.103312 22107 raft_consensus.cc:2272] T 5a57670b155f42dabe1699c849e90e92 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.105522 22107 tablet_replica.cc:333] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.106194 22107 raft_consensus.cc:2243] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.106863 22107 raft_consensus.cc:2272] T a3468b676b534142851875be523c6ee0 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.109253 22107 tablet_replica.cc:333] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.110250 22107 raft_consensus.cc:2243] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.111112 22107 raft_consensus.cc:2272] T 148682670bd0400bbe7e8e3545f7937d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.113212 22107 tablet_replica.cc:333] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.113827 22107 raft_consensus.cc:2243] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:17.114302 22107 raft_consensus.cc:2272] T 03c8c69d2f8c424799063791f4f92c1d P a6cb411c8bd34b8ca71f2df2d45e0458 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.116104 22107 tablet_replica.cc:333] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.116600 22107 raft_consensus.cc:2243] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:17.117542 22107 raft_consensus.cc:2272] T 7e59dba78ece4f629820bd43f26fd10a P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.119681 22107 tablet_replica.cc:333] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.120374 22107 raft_consensus.cc:2243] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:17.121559 22107 raft_consensus.cc:2272] T c19ef16b692341018afac81fbd8b24bc P a6cb411c8bd34b8ca71f2df2d45e0458 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:17.125154 22107 tablet_replica.cc:333] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458: stopping tablet replica
I20260709 12:12:17.125797 22107 raft_consensus.cc:2243] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:17.126827 22107 raft_consensus.cc:2272] T 2211c5e3441e4641be59f669903d8a42 P a6cb411c8bd34b8ca71f2df2d45e0458 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:17.145079 22354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from unknown peer (error 108)
W20260709 12:12:17.237732 22354 consensus_peers.cc:597] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:17.242486 22280 consensus_peers.cc:597] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:17.313663 22280 consensus_peers.cc:597] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:17.385916 22280 consensus_peers.cc:597] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:17.435514 22354 consensus_peers.cc:597] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:17.436095 22354 consensus_peers.cc:597] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 -> Peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Couldn't send request to peer a6cb411c8bd34b8ca71f2df2d45e0458. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:18.140590 22678 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.141165 22678 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.143507 22678 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
I20260709 12:12:18.144685 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
W20260709 12:12:18.148511 22354 leader_election.cc:336] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.148969 22354 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:18.150084 22678 raft_consensus.cc:2749] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:12:18.245887 22107 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22263)
I20260709 12:12:18.287787 22671 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.288493 22671 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:18.291510 22671 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:12:18.293061 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
W20260709 12:12:18.297473 22280 leader_election.cc:336] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.297966 22280 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536
I20260709 12:12:18.298885 22671 raft_consensus.cc:2749] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:12:18.473596 22677 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.474210 22677 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.476145 22677 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:18.477278 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2211c5e3441e4641be59f669903d8a42" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
W20260709 12:12:18.480185 22280 leader_election.cc:336] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.480590 22280 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, fc3e67a5db6c447583f943602f281536
I20260709 12:12:18.481372 22677 raft_consensus.cc:2749] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:18.485138 22677 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.485636 22677 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.487421 22677 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:18.488736 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536" is_pre_election: true
I20260709 12:12:18.489401 22390 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 4.
I20260709 12:12:18.491021 22281 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:18.492112 22677 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:12:18.492384 22280 leader_election.cc:336] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.492570 22677 raft_consensus.cc:493] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.492983 22677 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:12:18.502215 22677 raft_consensus.cc:515] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.504194 22677 leader_election.cc:290] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 election: Requested vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), fc3e67a5db6c447583f943602f281536 (127.21.150.195:46245)
I20260709 12:12:18.505520 22390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19ef16b692341018afac81fbd8b24bc" candidate_uuid: "cd4d6e7b1323429c8880fe673141ca39" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "fc3e67a5db6c447583f943602f281536"
I20260709 12:12:18.506105 22390 raft_consensus.cc:3060] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:18.508289 22280 leader_election.cc:336] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.511184 22390 raft_consensus.cc:2468] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4d6e7b1323429c8880fe673141ca39 in term 5.
I20260709 12:12:18.512116 22281 leader_election.cc:304] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458
I20260709 12:12:18.512796 22677 raft_consensus.cc:2804] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:12:18.513293 22677 raft_consensus.cc:697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 LEADER]: Becoming Leader. State: Replica: cd4d6e7b1323429c8880fe673141ca39, State: Running, Role: LEADER
I20260709 12:12:18.514196 22677 consensus_queue.cc:237] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.519691 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
I20260709 12:12:18.523511 22136 catalog_manager.cc:5697] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 reported cstate change: term changed from 4 to 5, leader changed from a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193) to cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "cd4d6e7b1323429c8880fe673141ca39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:18.542136 22678 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.542804 22678 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:18.545086 22678 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
I20260709 12:12:18.546667 22317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e59dba78ece4f629820bd43f26fd10a" candidate_uuid: "fc3e67a5db6c447583f943602f281536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd4d6e7b1323429c8880fe673141ca39" is_pre_election: true
I20260709 12:12:18.547271 22317 raft_consensus.cc:2468] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc3e67a5db6c447583f943602f281536 in term 4.
I20260709 12:12:18.548648 22357 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd4d6e7b1323429c8880fe673141ca39, fc3e67a5db6c447583f943602f281536; no voters: 
I20260709 12:12:18.549399 22678 raft_consensus.cc:2804] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:12:18.549820 22678 raft_consensus.cc:493] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.550210 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
I20260709 12:12:18.550275 22678 raft_consensus.cc:3060] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:12:18.550132 22354 leader_election.cc:336] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.561367 22678 raft_consensus.cc:515] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } }
I20260709 12:12:18.567359 22678 leader_election.cc:290] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: Requested vote from peers cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745), a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045)
W20260709 12:12:18.568807 22357 leader_election.cc:336] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:12:18.573787 22354 leader_election.cc:336] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.574358 22354 leader_election.cc:304] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:18.575062 22678 raft_consensus.cc:2749] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:12:18.585522 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:18.586453 22107 tablet_replica.cc:333] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.587380 22107 raft_consensus.cc:2243] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.588023 22107 raft_consensus.cc:2272] T 2211c5e3441e4641be59f669903d8a42 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.589901 22107 tablet_replica.cc:333] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.590421 22107 raft_consensus.cc:2243] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.591215 22107 raft_consensus.cc:2272] T 5a57670b155f42dabe1699c849e90e92 P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.593601 22107 tablet_replica.cc:333] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.594331 22107 raft_consensus.cc:2243] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.594935 22107 raft_consensus.cc:2272] T af9f742961e44504ab2db17c60a2730b P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:18.596936 22357 consensus_peers.cc:597] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 -> Peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Couldn't send request to peer cd4d6e7b1323429c8880fe673141ca39. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:18.597131 22107 tablet_replica.cc:333] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.597888 22107 raft_consensus.cc:2243] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.598891 22107 raft_consensus.cc:2272] T 59269a8cb6ec4500b34f4e7292005694 P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.600886 22107 tablet_replica.cc:333] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.601550 22107 raft_consensus.cc:2243] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.602365 22107 raft_consensus.cc:2272] T 03c8c69d2f8c424799063791f4f92c1d P cd4d6e7b1323429c8880fe673141ca39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.604189 22107 tablet_replica.cc:333] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.604733 22107 raft_consensus.cc:2243] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.605211 22107 raft_consensus.cc:2272] T 7e59dba78ece4f629820bd43f26fd10a P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.606941 22107 tablet_replica.cc:333] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.607528 22107 raft_consensus.cc:2243] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.608376 22107 raft_consensus.cc:2272] T 148682670bd0400bbe7e8e3545f7937d P cd4d6e7b1323429c8880fe673141ca39 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.610386 22107 tablet_replica.cc:333] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.611052 22107 raft_consensus.cc:2243] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.611550 22107 raft_consensus.cc:2272] T a3468b676b534142851875be523c6ee0 P cd4d6e7b1323429c8880fe673141ca39 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.613360 22107 tablet_replica.cc:333] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: stopping tablet replica
I20260709 12:12:18.613893 22107 raft_consensus.cc:2243] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.614512 22107 pending_rounds.cc:70] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Trying to abort 1 pending ops.
I20260709 12:12:18.614784 22107 pending_rounds.cc:77] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39: Aborting op as it isn't in flight: id { term: 5 index: 5 } timestamp: 7305622071361495040 op_type: NO_OP noop_request { }
I20260709 12:12:18.615175 22107 raft_consensus.cc:2889] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:12:18.615492 22107 raft_consensus.cc:2272] T c19ef16b692341018afac81fbd8b24bc P cd4d6e7b1323429c8880fe673141ca39 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.624747 22678 raft_consensus.cc:493] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a6cb411c8bd34b8ca71f2df2d45e0458)
I20260709 12:12:18.626192 22678 raft_consensus.cc:515] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6cb411c8bd34b8ca71f2df2d45e0458" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45045 } } peers { permanent_uuid: "cd4d6e7b1323429c8880fe673141ca39" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 37745 } } peers { permanent_uuid: "fc3e67a5db6c447583f943602f281536" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 46245 } }
I20260709 12:12:18.628795 22678 leader_election.cc:290] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045), cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745)
W20260709 12:12:18.632164 22281 connection.cc:466] server connection from 127.0.0.1:47396 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47396 (request call id 196) could send its response
W20260709 12:12:18.633445 22357 leader_election.cc:336] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Network error: recv got EOF from 127.21.150.194:37745 (error 108)
W20260709 12:12:18.634841 22354 leader_election.cc:336] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a6cb411c8bd34b8ca71f2df2d45e0458 (127.21.150.193:45045): Network error: Client connection negotiation failed: client connection to 127.21.150.193:45045: connect: Connection refused (error 111)
I20260709 12:12:18.635357 22354 leader_election.cc:304] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fc3e67a5db6c447583f943602f281536; no voters: a6cb411c8bd34b8ca71f2df2d45e0458, cd4d6e7b1323429c8880fe673141ca39
I20260709 12:12:18.636535 22678 raft_consensus.cc:2749] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:12:18.650043 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
I20260709 12:12:18.668565 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:12:18.694113 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:18.694837 22107 tablet_replica.cc:333] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.695574 22107 raft_consensus.cc:2243] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.696065 22107 raft_consensus.cc:2272] T 148682670bd0400bbe7e8e3545f7937d P fc3e67a5db6c447583f943602f281536 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.698320 22107 tablet_replica.cc:333] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.698823 22107 raft_consensus.cc:2243] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.699515 22107 raft_consensus.cc:2272] T a3468b676b534142851875be523c6ee0 P fc3e67a5db6c447583f943602f281536 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:18.699633 22357 consensus_peers.cc:597] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 -> Peer cd4d6e7b1323429c8880fe673141ca39 (127.21.150.194:37745): Couldn't send request to peer cd4d6e7b1323429c8880fe673141ca39. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:37745: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:18.701257 22107 tablet_replica.cc:333] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.701834 22107 raft_consensus.cc:2243] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.702557 22107 raft_consensus.cc:2272] T af9f742961e44504ab2db17c60a2730b P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.704216 22107 tablet_replica.cc:333] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.704764 22107 raft_consensus.cc:2243] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.705231 22107 raft_consensus.cc:2272] T 59269a8cb6ec4500b34f4e7292005694 P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.706833 22107 tablet_replica.cc:333] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.707312 22107 raft_consensus.cc:2243] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.707769 22107 raft_consensus.cc:2272] T 7e59dba78ece4f629820bd43f26fd10a P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.709576 22107 tablet_replica.cc:333] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.710052 22107 raft_consensus.cc:2243] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.710484 22107 raft_consensus.cc:2272] T 03c8c69d2f8c424799063791f4f92c1d P fc3e67a5db6c447583f943602f281536 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.712478 22107 tablet_replica.cc:333] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.713193 22107 raft_consensus.cc:2243] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.714015 22107 raft_consensus.cc:2272] T 5a57670b155f42dabe1699c849e90e92 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.715778 22107 tablet_replica.cc:333] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.716576 22107 raft_consensus.cc:2243] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.717124 22107 raft_consensus.cc:2272] T 2211c5e3441e4641be59f669903d8a42 P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.718889 22107 tablet_replica.cc:333] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536: stopping tablet replica
I20260709 12:12:18.719324 22107 raft_consensus.cc:2243] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:18.719722 22107 raft_consensus.cc:2272] T c19ef16b692341018afac81fbd8b24bc P fc3e67a5db6c447583f943602f281536 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.748383 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:12:18.767722 22107 master.cc:562] Master@127.21.150.254:33547 shutting down...
I20260709 12:12:18.925868 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:12:18.926499 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:18.926855 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P d3ccd713489c4d409eee5fdd03f3fae1: stopping tablet replica
I20260709 12:12:18.949489 22107 master.cc:584] Master@127.21.150.254:33547 shutdown complete.
a6cb411c8bd34b8ca71f2df2d45e0458  4
cd4d6e7b1323429c8880fe673141ca39  4
fc3e67a5db6c447583f943602f281536  1
a6cb411c8bd34b8ca71f2df2d45e0458  3
cd4d6e7b1323429c8880fe673141ca39  3
fc3e67a5db6c447583f943602f281536  3
a6cb411c8bd34b8ca71f2df2d45e0458  0
cd4d6e7b1323429c8880fe673141ca39  8
fc3e67a5db6c447583f943602f281536  1
a6cb411c8bd34b8ca71f2df2d45e0458  3
cd4d6e7b1323429c8880fe673141ca39  3
fc3e67a5db6c447583f943602f281536  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (34482 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:12:18.994660 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:39721
I20260709 12:12:18.995822 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:19.002696 22684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.002718 22685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.005498 22687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.006753 22107 server_base.cc:1061] running on GCE node
I20260709 12:12:19.008054 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:19.008287 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:19.008488 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599139008453 us; error 0 us; skew 500 ppm
I20260709 12:12:19.009038 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:19.011615 22107 webserver.cc:533] Webserver started at http://127.21.150.254:45505/ using document root <none> and password file <none>
I20260709 12:12:19.012187 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:19.012379 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:19.012683 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:19.014107 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "00abdabf4916465083aeda8a2667460e"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.018972 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:12:19.022704 22692 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.023715 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:12:19.023968 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "00abdabf4916465083aeda8a2667460e"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.024222 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:19.038436 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:19.039655 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:19.081318 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:39721
I20260709 12:12:19.081513 22743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:39721 every 8 connection(s)
I20260709 12:12:19.085270 22744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.095039 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e: Bootstrap starting.
I20260709 12:12:19.100390 22744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.106037 22744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e: No bootstrap required, opened a new log
I20260709 12:12:19.108408 22744 raft_consensus.cc:359] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER }
I20260709 12:12:19.108842 22744 raft_consensus.cc:385] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.109143 22744 raft_consensus.cc:740] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00abdabf4916465083aeda8a2667460e, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.109908 22744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER }
I20260709 12:12:19.110375 22744 raft_consensus.cc:399] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:12:19.110670 22744 raft_consensus.cc:493] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:12:19.110991 22744 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:19.116303 22744 raft_consensus.cc:515] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER }
I20260709 12:12:19.116932 22744 leader_election.cc:304] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00abdabf4916465083aeda8a2667460e; no voters: 
I20260709 12:12:19.118250 22744 leader_election.cc:290] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:12:19.118674 22747 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:19.120095 22747 raft_consensus.cc:697] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 1 LEADER]: Becoming Leader. State: Replica: 00abdabf4916465083aeda8a2667460e, State: Running, Role: LEADER
I20260709 12:12:19.120800 22747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER }
I20260709 12:12:19.121371 22744 sys_catalog.cc:565] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:12:19.125789 22749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00abdabf4916465083aeda8a2667460e. Latest consensus state: current_term: 1 leader_uuid: "00abdabf4916465083aeda8a2667460e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER } }
I20260709 12:12:19.126632 22749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:19.128508 22748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00abdabf4916465083aeda8a2667460e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00abdabf4916465083aeda8a2667460e" member_type: VOTER } }
I20260709 12:12:19.129321 22748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:19.133384 22755 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:12:19.138525 22755 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:12:19.140048 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:12:19.148770 22755 catalog_manager.cc:1383] Generated new cluster ID: f48975414140457586597508661ed15f
I20260709 12:12:19.149045 22755 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:12:19.164400 22755 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:12:19.165753 22755 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:12:19.190510 22755 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e: Generated new TSK 0
I20260709 12:12:19.191169 22755 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:12:19.207188 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:19.213701 22765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.215703 22766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.216836 22768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.218366 22107 server_base.cc:1061] running on GCE node
I20260709 12:12:19.219368 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:19.219564 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:19.219905 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599139219886 us; error 0 us; skew 500 ppm
I20260709 12:12:19.220666 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:19.223417 22107 webserver.cc:533] Webserver started at http://127.21.150.193:38495/ using document root <none> and password file <none>
I20260709 12:12:19.223944 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:19.224118 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:19.224438 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:19.225648 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "872c1e6c20d44893bb11247b8ba40760"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.230402 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:12:19.234454 22773 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.235769 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:12:19.236032 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "872c1e6c20d44893bb11247b8ba40760"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.236289 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:19.253752 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:19.255098 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:19.256650 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:19.259227 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:19.259419 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.259747 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:19.260337 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.307895 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:33155
I20260709 12:12:19.308002 22835 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:33155 every 8 connection(s)
I20260709 12:12:19.313097 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:19.320039 22840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.321679 22841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.324394 22107 server_base.cc:1061] running on GCE node
W20260709 12:12:19.325649 22843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.326287 22836 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39721
I20260709 12:12:19.326683 22836 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:19.326721 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:19.327030 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:19.327209 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599139327177 us; error 0 us; skew 500 ppm
I20260709 12:12:19.327574 22836 heartbeater.cc:507] Master 127.21.150.254:39721 requested a full tablet report, sending...
I20260709 12:12:19.327780 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:19.329928 22709 ts_manager.cc:194] Registered new tserver with Master: 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155)
I20260709 12:12:19.330545 22107 webserver.cc:533] Webserver started at http://127.21.150.194:35621/ using document root <none> and password file <none>
I20260709 12:12:19.331065 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:19.331286 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:19.331593 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:19.332018 22709 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55316
I20260709 12:12:19.333135 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "e5dd675310a7430aa685001cdd0ff16b"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.337569 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:12:19.341107 22848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.341933 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:12:19.342271 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "e5dd675310a7430aa685001cdd0ff16b"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.342576 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:19.359620 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:19.360913 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:19.362522 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:19.364750 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:19.364984 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.365249 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:19.365447 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.409546 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:40991
I20260709 12:12:19.409690 22910 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:40991 every 8 connection(s)
I20260709 12:12:19.414304 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:19.421970 22914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:19.424037 22915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.426034 22107 server_base.cc:1061] running on GCE node
W20260709 12:12:19.426239 22917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:19.427224 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:19.427455 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:19.427636 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599139427619 us; error 0 us; skew 500 ppm
I20260709 12:12:19.428166 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:19.430497 22107 webserver.cc:533] Webserver started at http://127.21.150.195:35223/ using document root <none> and password file <none>
I20260709 12:12:19.430998 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:19.431214 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:19.431481 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:19.432643 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.437815 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:12:19.439210 22911 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39721
I20260709 12:12:19.439639 22911 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:19.440541 22911 heartbeater.cc:507] Master 127.21.150.254:39721 requested a full tablet report, sending...
I20260709 12:12:19.441877 22922 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.442843 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:12:19.443054 22709 ts_manager.cc:194] Registered new tserver with Master: e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:19.443207 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
format_stamp: "Formatted at 2026-07-09 12:12:19 on dist-test-slave-zctp"
I20260709 12:12:19.443617 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:19.444892 22709 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55326
I20260709 12:12:19.460309 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:19.461585 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:19.463043 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:19.465282 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:19.465477 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:12:19.465785 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:19.465942 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:19.512471 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:42245
I20260709 12:12:19.512684 22984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:42245 every 8 connection(s)
I20260709 12:12:19.528419 22985 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39721
I20260709 12:12:19.528834 22985 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:19.529630 22985 heartbeater.cc:507] Master 127.21.150.254:39721 requested a full tablet report, sending...
I20260709 12:12:19.531606 22709 ts_manager.cc:194] Registered new tserver with Master: 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:19.532414 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016026148s
I20260709 12:12:19.532939 22709 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55340
I20260709 12:12:19.560344 22709 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55354:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:12:19.563392 22709 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:12:19.636941 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 4afe3dc6a22149639feac8d506b61d75 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:19.638026 22801 tablet_service.cc:1511] Processing CreateTablet for tablet 4afe3dc6a22149639feac8d506b61d75 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:19.638608 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4afe3dc6a22149639feac8d506b61d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.639397 22801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4afe3dc6a22149639feac8d506b61d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.646205 22875 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5ea0116fc44819c4e341f9ec4069b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:19.647774 22875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5ea0116fc44819c4e341f9ec4069b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.648918 22800 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5ea0116fc44819c4e341f9ec4069b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:19.650486 22800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5ea0116fc44819c4e341f9ec4069b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.659716 22874 tablet_service.cc:1511] Processing CreateTablet for tablet 77a2992f247b4a41b3affc19472e2d36 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:19.661396 22874 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a2992f247b4a41b3affc19472e2d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.665966 22798 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca67b03f1ef4e4fbf649384cb9d8378 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:19.667333 22798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca67b03f1ef4e4fbf649384cb9d8378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.662871 22799 tablet_service.cc:1511] Processing CreateTablet for tablet 77a2992f247b4a41b3affc19472e2d36 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:19.670526 22799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a2992f247b4a41b3affc19472e2d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.663136 22873 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca67b03f1ef4e4fbf649384cb9d8378 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:19.674556 22873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca67b03f1ef4e4fbf649384cb9d8378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.679350 22872 tablet_service.cc:1511] Processing CreateTablet for tablet 725faa936c0c4ac8ac9696dfaa1470de (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:19.681094 22797 tablet_service.cc:1511] Processing CreateTablet for tablet 725faa936c0c4ac8ac9696dfaa1470de (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:19.682476 22797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725faa936c0c4ac8ac9696dfaa1470de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.688062 22795 tablet_service.cc:1511] Processing CreateTablet for tablet 6216945e97b14cffba0ece229f0ed12b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:19.691465 23000 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:19.691677 22949 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5ea0116fc44819c4e341f9ec4069b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:12:19.693162 22949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5ea0116fc44819c4e341f9ec4069b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.695063 22947 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca67b03f1ef4e4fbf649384cb9d8378 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:12:19.697019 22947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca67b03f1ef4e4fbf649384cb9d8378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.697163 22876 tablet_service.cc:1511] Processing CreateTablet for tablet 844a19913c634ce7b8fba9c809990942 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:19.698764 22946 tablet_service.cc:1511] Processing CreateTablet for tablet 725faa936c0c4ac8ac9696dfaa1470de (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:12:19.700176 22946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725faa936c0c4ac8ac9696dfaa1470de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.700398 22871 tablet_service.cc:1511] Processing CreateTablet for tablet 6216945e97b14cffba0ece229f0ed12b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:19.700971 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 4badfdfaa1b04d92a3ace654c6a7397c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:19.702844 22945 tablet_service.cc:1511] Processing CreateTablet for tablet 844a19913c634ce7b8fba9c809990942 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:19.704309 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a19913c634ce7b8fba9c809990942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.706813 22944 tablet_service.cc:1511] Processing CreateTablet for tablet 6216945e97b14cffba0ece229f0ed12b (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:12:19.708448 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6216945e97b14cffba0ece229f0ed12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.710896 22948 tablet_service.cc:1511] Processing CreateTablet for tablet 77a2992f247b4a41b3affc19472e2d36 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:12:19.711686 22869 tablet_service.cc:1511] Processing CreateTablet for tablet db4907f4b11a455c9ac45cf80d6fbe8c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:19.712558 22948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77a2992f247b4a41b3affc19472e2d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.714746 22794 tablet_service.cc:1511] Processing CreateTablet for tablet 4badfdfaa1b04d92a3ace654c6a7397c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:19.681262 22872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725faa936c0c4ac8ac9696dfaa1470de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.718809 22943 tablet_service.cc:1511] Processing CreateTablet for tablet 4badfdfaa1b04d92a3ace654c6a7397c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:12:19.716234 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 864a8e326a9944a5a8fed670ce1a84ec (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:19.735075 22795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6216945e97b14cffba0ece229f0ed12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.739625 22950 tablet_service.cc:1511] Processing CreateTablet for tablet 4afe3dc6a22149639feac8d506b61d75 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:12:19.742352 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4badfdfaa1b04d92a3ace654c6a7397c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.745937 22876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a19913c634ce7b8fba9c809990942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.746517 22800 tablet_service.cc:1511] Processing CreateTablet for tablet db4907f4b11a455c9ac45cf80d6fbe8c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:19.747741 22800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4907f4b11a455c9ac45cf80d6fbe8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.750695 22793 tablet_service.cc:1511] Processing CreateTablet for tablet 864a8e326a9944a5a8fed670ce1a84ec (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:19.752048 22793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a8e326a9944a5a8fed670ce1a84ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.754004 22871 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6216945e97b14cffba0ece229f0ed12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.758432 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a8e326a9944a5a8fed670ce1a84ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.758581 22942 tablet_service.cc:1511] Processing CreateTablet for tablet db4907f4b11a455c9ac45cf80d6fbe8c (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:12:19.760680 22794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4badfdfaa1b04d92a3ace654c6a7397c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.686692 22796 tablet_service.cc:1511] Processing CreateTablet for tablet 844a19913c634ce7b8fba9c809990942 (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:12:19.763053 22796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 844a19913c634ce7b8fba9c809990942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.765812 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4badfdfaa1b04d92a3ace654c6a7397c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.775501 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4907f4b11a455c9ac45cf80d6fbe8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.778231 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db4907f4b11a455c9ac45cf80d6fbe8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.778893 22941 tablet_service.cc:1511] Processing CreateTablet for tablet 864a8e326a9944a5a8fed670ce1a84ec (DEFAULT_TABLE table=test-workload [id=063580f2d08c4841aec4869ec2061c47]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:12:19.782632 22950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4afe3dc6a22149639feac8d506b61d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.783383 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 864a8e326a9944a5a8fed670ce1a84ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:19.790254 23002 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:19.803679 23001 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:19.830557 23000 tablet_bootstrap.cc:654] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.858875 23002 tablet_bootstrap.cc:654] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.889591 23001 tablet_bootstrap.cc:654] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.899394 23000 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:19.899839 23000 ts_tablet_manager.cc:1403] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.209s	user 0.030s	sys 0.007s
I20260709 12:12:19.901051 23002 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:19.901791 23002 ts_tablet_manager.cc:1403] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.112s	user 0.019s	sys 0.006s
I20260709 12:12:19.905979 23002 raft_consensus.cc:359] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.907411 23002 raft_consensus.cc:385] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.908063 23002 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.909255 23002 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.912550 23002 ts_tablet_manager.cc:1434] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:12:19.913203 23001 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:19.913278 22985 heartbeater.cc:499] Master 127.21.150.254:39721 was elected leader, sending a full tablet report...
I20260709 12:12:19.909657 23000 raft_consensus.cc:359] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.914063 23000 raft_consensus.cc:385] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.914418 23002 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:19.913789 23001 ts_tablet_manager.cc:1403] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.012s
I20260709 12:12:19.914444 23000 raft_consensus.cc:740] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.916064 23000 consensus_queue.cc:260] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.918365 23001 raft_consensus.cc:359] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.919138 23001 raft_consensus.cc:385] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.919437 23001 raft_consensus.cc:740] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.920301 23001 consensus_queue.cc:260] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.922569 23002 tablet_bootstrap.cc:654] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.926213 22836 heartbeater.cc:499] Master 127.21.150.254:39721 was elected leader, sending a full tablet report...
I20260709 12:12:19.927379 23001 ts_tablet_manager.cc:1434] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
I20260709 12:12:19.928544 23001 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:19.935508 23001 tablet_bootstrap.cc:654] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.940469 23007 raft_consensus.cc:493] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.941066 23007 raft_consensus.cc:515] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.941747 22911 heartbeater.cc:499] Master 127.21.150.254:39721 was elected leader, sending a full tablet report...
I20260709 12:12:19.946944 23000 ts_tablet_manager.cc:1434] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.047s	user 0.012s	sys 0.004s
I20260709 12:12:19.946553 23007 leader_election.cc:290] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:19.949779 23000 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:19.959754 23001 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:19.960234 23001 ts_tablet_manager.cc:1403] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 12:12:19.962779 23001 raft_consensus.cc:359] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.963527 23001 raft_consensus.cc:385] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.963845 23001 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.964622 23001 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.966816 23001 ts_tablet_manager.cc:1434] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:12:19.967747 23001 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:19.968218 23000 tablet_bootstrap.cc:654] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.973745 23002 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:19.974246 23002 ts_tablet_manager.cc:1403] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.020s
I20260709 12:12:19.974457 23001 tablet_bootstrap.cc:654] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.977280 23002 raft_consensus.cc:359] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.978287 23002 raft_consensus.cc:385] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:19.978632 23002 raft_consensus.cc:740] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:19.980218 23002 consensus_queue.cc:260] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:19.982807 23002 ts_tablet_manager.cc:1434] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:12:19.983804 23002 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:19.992197 23002 tablet_bootstrap.cc:654] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:19.997193 23001 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:19.997895 23001 ts_tablet_manager.cc:1403] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.003s
I20260709 12:12:19.999572 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.000367 22811 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 0.
I20260709 12:12:20.001693 23001 raft_consensus.cc:359] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.002089 22852 leader_election.cc:304] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.002677 23001 raft_consensus.cc:385] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.003103 23001 raft_consensus.cc:740] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.003396 23010 raft_consensus.cc:2804] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.003904 23010 raft_consensus.cc:493] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.004258 23010 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.004278 23001 consensus_queue.cc:260] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.012208 23000 tablet_bootstrap.cc:492] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.012758 23000 ts_tablet_manager.cc:1403] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.004s
I20260709 12:12:20.014508 23010 raft_consensus.cc:515] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.017644 23000 raft_consensus.cc:359] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.018419 23000 raft_consensus.cc:385] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:12:20.018669 22986 tablet.cc:2406] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:12:20.018754 23000 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.019600 23000 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.021018 23001 ts_tablet_manager.cc:1434] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:12:20.022225 23000 ts_tablet_manager.cc:1434] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:12:20.022395 23001 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.023176 23000 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.028296 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.028934 22811 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.030805 23000 tablet_bootstrap.cc:654] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.032735 23001 tablet_bootstrap.cc:654] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.043236 23002 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.051568 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.043932 23002 ts_tablet_manager.cc:1403] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.060s	user 0.023s	sys 0.019s
I20260709 12:12:20.052697 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
W20260709 12:12:20.057662 22852 leader_election.cc:343] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:20.058634 22852 leader_election.cc:343] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.047283 23010 leader_election.cc:290] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.059201 23002 raft_consensus.cc:359] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.059898 23002 raft_consensus.cc:385] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.060055 23000 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.060241 23002 raft_consensus.cc:740] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.060596 23000 ts_tablet_manager.cc:1403] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.016s
I20260709 12:12:20.061494 23001 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.061046 23002 consensus_queue.cc:260] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.061947 23001 ts_tablet_manager.cc:1403] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.015s
I20260709 12:12:20.063576 23002 ts_tablet_manager.cc:1434] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:20.064832 23002 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.064898 23000 raft_consensus.cc:359] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.044122 22811 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 1.
I20260709 12:12:20.065236 23001 raft_consensus.cc:359] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.065819 23000 raft_consensus.cc:385] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.066061 23001 raft_consensus.cc:385] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.066241 23000 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.066529 23001 raft_consensus.cc:740] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.067252 23000 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.067436 23001 consensus_queue.cc:260] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.068826 22852 leader_election.cc:304] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:20.069945 23001 ts_tablet_manager.cc:1434] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:12:20.070250 23007 raft_consensus.cc:2804] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.070995 23001 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.073091 23002 tablet_bootstrap.cc:654] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.076603 23000 ts_tablet_manager.cc:1434] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:12:20.077683 23000 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.077956 23001 tablet_bootstrap.cc:654] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.084021 23000 tablet_bootstrap.cc:654] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.084944 23002 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.085592 23002 ts_tablet_manager.cc:1403] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.021s	user 0.020s	sys 0.000s
I20260709 12:12:20.088584 23002 raft_consensus.cc:359] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.089334 23002 raft_consensus.cc:385] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.089668 23002 raft_consensus.cc:740] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.090549 23002 consensus_queue.cc:260] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.092048 23001 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.092540 23001 ts_tablet_manager.cc:1403] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 12:12:20.092811 23002 ts_tablet_manager.cc:1434] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:20.093931 23002 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.095453 23001 raft_consensus.cc:359] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.096186 23001 raft_consensus.cc:385] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.096473 23001 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.097503 23001 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.100756 23002 tablet_bootstrap.cc:654] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.102732 23001 ts_tablet_manager.cc:1434] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:12:20.104135 23001 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.119335 23001 tablet_bootstrap.cc:654] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.119699 23006 raft_consensus.cc:493] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.120862 23006 raft_consensus.cc:515] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.125638 23006 leader_election.cc:290] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.126976 23006 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.127543 23006 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
W20260709 12:12:20.134470 22757 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:20.135637 23002 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.128909 23007 raft_consensus.cc:697] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:20.137179 23006 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.136786 23007 consensus_queue.cc:237] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.139426 23000 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.139887 23000 ts_tablet_manager.cc:1403] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.062s	user 0.029s	sys 0.022s
I20260709 12:12:20.142570 23000 raft_consensus.cc:359] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.144821 23002 ts_tablet_manager.cc:1403] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.012s
I20260709 12:12:20.144181 23008 raft_consensus.cc:493] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.143517 23000 raft_consensus.cc:385] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.147588 23000 raft_consensus.cc:740] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.146548 23008 raft_consensus.cc:515] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.148548 23000 consensus_queue.cc:260] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.150151 23002 raft_consensus.cc:359] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.151006 23002 raft_consensus.cc:385] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.151400 23000 ts_tablet_manager.cc:1434] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 12:12:20.151355 23002 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.151315 22707 catalog_manager.cc:5697] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 0 to 1, leader changed from <none> to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 1 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.152683 23000 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.152482 23002 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.159693 23008 leader_election.cc:290] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.164718 23001 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.165758 23001 ts_tablet_manager.cc:1403] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.006s
I20260709 12:12:20.166265 23008 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.166762 23008 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.168215 23000 tablet_bootstrap.cc:654] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.168879 23008 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.173679 23001 raft_consensus.cc:359] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.174703 23001 raft_consensus.cc:385] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.180433 23002 ts_tablet_manager.cc:1434] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.003s
I20260709 12:12:20.185093 23002 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.192353 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
W20260709 12:12:20.197190 22775 leader_election.cc:343] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.197507 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.193598 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.198352 22885 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:20.194380 23008 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.199430 23008 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
W20260709 12:12:20.199949 22926 leader_election.cc:343] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.198824 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.200040 22775 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.200739 22810 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 0.
I20260709 12:12:20.175134 23001 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.206470 23008 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.206576 22926 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.207893 23006 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.208408 23006 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.208842 23006 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.207548 23008 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.214205 23008 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.214763 23008 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.221939 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.221410 23001 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.225147 23001 ts_tablet_manager.cc:1434] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.004s
I20260709 12:12:20.230796 23001 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.231334 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.231835 22886 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:20.232890 22775 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.233150 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.234473 22885 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 0.
W20260709 12:12:20.238380 22924 leader_election.cc:343] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.239163 23001 tablet_bootstrap.cc:654] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.240134 23033 raft_consensus.cc:2804] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.240595 23033 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.241011 23033 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.238886 22924 leader_election.cc:304] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a; no voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:20.248883 23002 tablet_bootstrap.cc:654] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.255730 23034 raft_consensus.cc:2749] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:20.256440 23034 raft_consensus.cc:493] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.257196 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.256938 23034 raft_consensus.cc:515] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.257334 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.259057 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
W20260709 12:12:20.260106 22777 leader_election.cc:343] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.260215 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.260372 23034 leader_election.cc:290] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.261029 22810 raft_consensus.cc:2468] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 0.
I20260709 12:12:20.260903 22777 leader_election.cc:304] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:20.262493 22926 leader_election.cc:304] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.263682 23034 raft_consensus.cc:2804] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.264187 23034 raft_consensus.cc:493] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.264178 23033 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.264938 23034 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.237753 23006 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.272276 23008 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.273295 23000 tablet_bootstrap.cc:492] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.273615 23006 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.273965 23000 ts_tablet_manager.cc:1403] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.122s	user 0.036s	sys 0.032s
I20260709 12:12:20.274580 23008 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.275136 23008 raft_consensus.cc:2749] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:20.275840 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.276814 22810 raft_consensus.cc:2393] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in current term 1: Already voted for candidate 872c1e6c20d44893bb11247b8ba40760 in this term.
I20260709 12:12:20.274600 22959 raft_consensus.cc:2393] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 872c1e6c20d44893bb11247b8ba40760 in current term 1: Already voted for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in this term.
I20260709 12:12:20.269996 23033 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
W20260709 12:12:20.286579 22775 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:40991, user_credentials={real_user=slave}} blocked reactor thread for 53775.1us
W20260709 12:12:20.287364 22777 leader_election.cc:343] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.288746 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.289566 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.290143 22886 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.290210 23000 raft_consensus.cc:359] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.291627 23000 raft_consensus.cc:385] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.292040 23000 raft_consensus.cc:740] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.293203 23000 consensus_queue.cc:260] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.302237 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.305625 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.306756 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.308055 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.308383 22885 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.308944 22959 raft_consensus.cc:2393] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 872c1e6c20d44893bb11247b8ba40760 in current term 1: Already voted for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in this term.
I20260709 12:12:20.307209 22886 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 1.
W20260709 12:12:20.310122 22924 leader_election.cc:343] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.311213 23001 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.311540 23001 ts_tablet_manager.cc:1403] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.081s	user 0.015s	sys 0.016s
I20260709 12:12:20.311590 22924 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:20.313470 23000 ts_tablet_manager.cc:1434] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
W20260709 12:12:20.308457 22777 leader_election.cc:343] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.314577 23000 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.314633 22775 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:20.315675 23035 raft_consensus.cc:2749] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:20.316537 23036 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.317080 23036 raft_consensus.cc:697] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:20.316996 23034 raft_consensus.cc:515] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.316927 23001 raft_consensus.cc:359] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.318847 23001 raft_consensus.cc:385] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.318521 22885 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:20.319537 23001 raft_consensus.cc:740] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.320770 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.320837 23001 consensus_queue.cc:260] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.321480 22810 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.323158 22775 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:20.324481 23034 leader_election.cc:290] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.318013 23036 consensus_queue.cc:237] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.325778 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.328169 22885 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.331094 23002 tablet_bootstrap.cc:492] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.331768 23002 ts_tablet_manager.cc:1403] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.147s	user 0.022s	sys 0.017s
I20260709 12:12:20.331147 23035 raft_consensus.cc:2804] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.333005 23035 raft_consensus.cc:697] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:20.334131 23035 consensus_queue.cc:237] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.335263 22885 raft_consensus.cc:2468] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 1.
I20260709 12:12:20.336971 23002 raft_consensus.cc:359] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.337693 23002 raft_consensus.cc:385] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.337993 23002 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.338718 23002 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.340235 23001 ts_tablet_manager.cc:1434] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:12:20.341605 23001 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.341734 22810 raft_consensus.cc:2468] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 1.
I20260709 12:12:20.343067 22926 leader_election.cc:304] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.344457 23034 raft_consensus.cc:2804] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.344931 23034 raft_consensus.cc:697] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:20.345778 23034 consensus_queue.cc:237] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.346612 23000 tablet_bootstrap.cc:654] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.352918 23002 ts_tablet_manager.cc:1434] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:12:20.354110 23002 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.357646 23000 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.358285 23000 ts_tablet_manager.cc:1403] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.003s
I20260709 12:12:20.360929 23002 tablet_bootstrap.cc:654] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.361331 23000 raft_consensus.cc:359] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.362157 23000 raft_consensus.cc:385] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.362463 23000 raft_consensus.cc:740] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.363497 23000 consensus_queue.cc:260] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.366276 23001 tablet_bootstrap.cc:654] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.363749 22707 catalog_manager.cc:5697] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 0 to 1, leader changed from <none> to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.370625 23000 ts_tablet_manager.cc:1434] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 12:12:20.370047 22709 catalog_manager.cc:5697] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 1 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:20.372601 23000 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.381110 23000 tablet_bootstrap.cc:654] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.384912 23002 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.385782 23002 ts_tablet_manager.cc:1403] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.014s
I20260709 12:12:20.393517 23007 raft_consensus.cc:493] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.393354 23002 raft_consensus.cc:359] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.394181 23002 raft_consensus.cc:385] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.394157 23007 raft_consensus.cc:515] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.394611 23002 raft_consensus.cc:740] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.395655 23002 consensus_queue.cc:260] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.396503 23007 leader_election.cc:290] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.398154 23002 ts_tablet_manager.cc:1434] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:12:20.400125 23002 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.399065 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.401165 22959 raft_consensus.cc:2468] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 0.
I20260709 12:12:20.402884 22852 leader_election.cc:304] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.404618 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
W20260709 12:12:20.406651 22852 leader_election.cc:343] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.407882 23019 raft_consensus.cc:2804] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.407795 23007 raft_consensus.cc:493] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.408421 23019 raft_consensus.cc:493] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.408823 23019 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.409180 23002 tablet_bootstrap.cc:654] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.408624 23007 raft_consensus.cc:515] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.414083 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.414263 23007 leader_election.cc:290] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.415660 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
W20260709 12:12:20.419337 22852 leader_election.cc:343] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:20.420276 22852 leader_election.cc:343] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.419564 22709 catalog_manager.cc:5697] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 1 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:20.420992 22852 leader_election.cc:304] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5dd675310a7430aa685001cdd0ff16b; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:20.423977 23007 raft_consensus.cc:2749] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:20.426210 23019 raft_consensus.cc:515] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.433533 23001 tablet_bootstrap.cc:492] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.433624 23019 leader_election.cc:290] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.434170 23001 ts_tablet_manager.cc:1403] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.093s	user 0.027s	sys 0.003s
I20260709 12:12:20.434548 23000 tablet_bootstrap.cc:492] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.435155 23000 ts_tablet_manager.cc:1403] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.063s	user 0.024s	sys 0.028s
I20260709 12:12:20.435184 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.435949 22959 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.438186 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.439298 23000 raft_consensus.cc:359] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.440285 23000 raft_consensus.cc:385] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.440675 23000 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.441742 23000 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.436890 23001 raft_consensus.cc:359] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.443095 23001 raft_consensus.cc:385] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.443375 23001 raft_consensus.cc:740] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.444335 23001 consensus_queue.cc:260] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.444897 23000 ts_tablet_manager.cc:1434] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:12:20.445158 22810 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.446416 23000 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.449815 22959 raft_consensus.cc:2468] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 1.
I20260709 12:12:20.452567 22852 leader_election.cc:304] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.453055 23002 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.453548 23002 ts_tablet_manager.cc:1403] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.017s
I20260709 12:12:20.453480 23019 raft_consensus.cc:2804] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.454198 23000 tablet_bootstrap.cc:654] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.454138 23019 raft_consensus.cc:697] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:20.455050 23019 consensus_queue.cc:237] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.459997 23000 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.460122 22810 raft_consensus.cc:2468] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 1.
I20260709 12:12:20.460462 23000 ts_tablet_manager.cc:1403] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:12:20.463032 23000 raft_consensus.cc:359] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.464099 23000 raft_consensus.cc:385] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.464367 23000 raft_consensus.cc:740] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.464977 23001 ts_tablet_manager.cc:1434] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 12:12:20.464668 23002 raft_consensus.cc:359] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.466519 23002 raft_consensus.cc:385] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.465708 22708 catalog_manager.cc:5697] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 0 to 1, leader changed from <none> to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 1 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.466845 23002 raft_consensus.cc:740] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.466444 23000 consensus_queue.cc:260] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.466972 23001 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.467923 23002 consensus_queue.cc:260] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.468773 23000 ts_tablet_manager.cc:1434] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:12:20.469728 23000 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.470173 23002 ts_tablet_manager.cc:1434] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:12:20.471014 23002 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.473246 23001 tablet_bootstrap.cc:654] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.475672 23000 tablet_bootstrap.cc:654] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.476315 23002 tablet_bootstrap.cc:654] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.480878 23036 raft_consensus.cc:493] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.481539 23036 raft_consensus.cc:515] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.485004 23035 raft_consensus.cc:493] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.485517 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.486424 22810 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 0.
I20260709 12:12:20.488281 22926 leader_election.cc:304] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.485486 23035 raft_consensus.cc:515] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.489269 23034 raft_consensus.cc:2804] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.489650 23034 raft_consensus.cc:493] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.489975 23034 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.491139 23035 leader_election.cc:290] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.492686 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.493934 22959 raft_consensus.cc:2468] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:20.494364 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.494381 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.495947 22777 leader_election.cc:304] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.497094 23035 raft_consensus.cc:2804] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.497481 23035 raft_consensus.cc:493] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.495311 22886 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 0.
I20260709 12:12:20.497859 23035 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:20.497717 22775 leader_election.cc:343] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:20.501068 23000 tablet_bootstrap.cc:492] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.501701 23000 ts_tablet_manager.cc:1403] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.004s
I20260709 12:12:20.492882 23036 leader_election.cc:290] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.506135 23035 raft_consensus.cc:515] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.500690 23034 raft_consensus.cc:515] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.508996 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.509313 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.509243 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.509905 22959 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.509995 22810 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.514909 22959 raft_consensus.cc:2468] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:20.515431 23001 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.516145 23001 ts_tablet_manager.cc:1403] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.016s
I20260709 12:12:20.516016 22777 leader_election.cc:304] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.516877 23033 raft_consensus.cc:2804] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.517341 23033 raft_consensus.cc:697] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:20.517526 23035 leader_election.cc:290] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.517607 22810 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 1.
I20260709 12:12:20.518182 23033 consensus_queue.cc:237] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.518985 22926 leader_election.cc:304] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:20.519507 23001 raft_consensus.cc:359] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.520156 23001 raft_consensus.cc:385] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.505311 23000 raft_consensus.cc:359] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.520776 23001 raft_consensus.cc:740] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.521287 23000 raft_consensus.cc:385] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.520670 23034 leader_election.cc:290] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.521682 23001 consensus_queue.cc:260] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.522264 23002 tablet_bootstrap.cc:492] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.522370 23034 raft_consensus.cc:2804] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.522418 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.522887 23002 ts_tablet_manager.cc:1403] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.011s
I20260709 12:12:20.523329 22885 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.524181 23001 ts_tablet_manager.cc:1434] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:12:20.521675 23000 raft_consensus.cc:740] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.525076 23001 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: Bootstrap starting.
I20260709 12:12:20.525174 23000 consensus_queue.cc:260] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.526152 22886 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.525713 23002 raft_consensus.cc:359] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.526569 23002 raft_consensus.cc:385] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.526827 23002 raft_consensus.cc:740] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.527585 23002 consensus_queue.cc:260] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.529796 23002 ts_tablet_manager.cc:1434] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:20.531332 23001 tablet_bootstrap.cc:654] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.531658 22885 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 1.
I20260709 12:12:20.532685 22886 raft_consensus.cc:2468] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:20.534266 23002 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Bootstrap starting.
I20260709 12:12:20.524084 23034 raft_consensus.cc:697] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:20.535210 23034 consensus_queue.cc:237] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.537216 23000 ts_tablet_manager.cc:1434] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:12:20.539808 23000 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: Bootstrap starting.
I20260709 12:12:20.547459 23000 tablet_bootstrap.cc:654] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.550446 22708 catalog_manager.cc:5697] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 0 to 1, leader changed from <none> to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.555567 23001 tablet_bootstrap.cc:492] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: No bootstrap required, opened a new log
I20260709 12:12:20.556190 23001 ts_tablet_manager.cc:1403] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.020s
I20260709 12:12:20.559485 23001 raft_consensus.cc:359] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.560361 23001 raft_consensus.cc:385] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.560704 23001 raft_consensus.cc:740] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.561769 23001 consensus_queue.cc:260] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.564837 23001 ts_tablet_manager.cc:1434] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:12:20.583271 23002 tablet_bootstrap.cc:654] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:20.586756 23000 tablet_bootstrap.cc:492] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: No bootstrap required, opened a new log
I20260709 12:12:20.587210 23000 ts_tablet_manager.cc:1403] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: Time spent bootstrapping tablet: real 0.048s	user 0.037s	sys 0.006s
I20260709 12:12:20.587597 22709 catalog_manager.cc:5697] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 1 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:20.589740 23002 tablet_bootstrap.cc:492] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: No bootstrap required, opened a new log
I20260709 12:12:20.590209 23002 ts_tablet_manager.cc:1403] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.003s
I20260709 12:12:20.590164 23019 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:20.593010 23002 raft_consensus.cc:359] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.593680 23002 raft_consensus.cc:385] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.597939 23002 raft_consensus.cc:740] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.597045 23000 raft_consensus.cc:359] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.598603 23002 consensus_queue.cc:260] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.599493 23000 raft_consensus.cc:385] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:20.599795 23000 raft_consensus.cc:740] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Initialized, Role: FOLLOWER
I20260709 12:12:20.598590 22959 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.600371 23000 consensus_queue.cc:260] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.601099 23002 ts_tablet_manager.cc:1434] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:12:20.602363 23000 ts_tablet_manager.cc:1434] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 12:12:20.610909 23019 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.615696 23034 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.616194 23034 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.618700 23034 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:20.619364 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.620182 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.620858 22885 raft_consensus.cc:2393] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in current term 1: Already voted for candidate 872c1e6c20d44893bb11247b8ba40760 in this term.
I20260709 12:12:20.621857 22924 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a; no voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:20.622702 23034 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.631039 23034 raft_consensus.cc:2749] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:20.663251 23019 raft_consensus.cc:493] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.663882 23019 raft_consensus.cc:515] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.665614 23019 leader_election.cc:290] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.666715 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760" is_pre_election: true
I20260709 12:12:20.666838 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.667477 22810 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 0.
I20260709 12:12:20.667477 22960 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 0.
I20260709 12:12:20.669027 22852 leader_election.cc:304] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.669991 23019 raft_consensus.cc:2804] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.670464 23019 raft_consensus.cc:493] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.670840 23019 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.675941 23019 raft_consensus.cc:515] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.677814 23019 leader_election.cc:290] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.678684 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:20.678854 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.679255 22810 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.679386 22960 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.684149 22960 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 1.
I20260709 12:12:20.684178 22810 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 1.
I20260709 12:12:20.685370 22852 leader_election.cc:304] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.686504 23019 raft_consensus.cc:2804] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.687089 23019 raft_consensus.cc:697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:20.687940 23019 consensus_queue.cc:237] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.696089 22708 catalog_manager.cc:5697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 0 to 1, leader changed from <none> to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 1 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.774966 23034 consensus_queue.cc:1048] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.788340 23006 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.790094 23006 consensus_queue.cc:1048] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.793010 23008 raft_consensus.cc:493] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.793655 23008 raft_consensus.cc:515] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.799054 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:20.799722 22886 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:20.797096 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:20.800909 22960 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:20.801137 22775 leader_election.cc:304] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.802454 23033 raft_consensus.cc:2804] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:20.802805 23033 raft_consensus.cc:493] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.803119 23033 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.804474 23036 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.805807 23035 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.811846 23033 raft_consensus.cc:515] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.812462 23008 leader_election.cc:290] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.815740 23033 leader_election.cc:290] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:20.816386 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:20.816999 22884 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.817039 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:20.817838 22960 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:20.826474 22884 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:20.831640 23033 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:20.831832 22775 leader_election.cc:304] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:20.832767 23035 raft_consensus.cc:2804] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:20.838567 23019 consensus_queue.cc:1048] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.842846 22960 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:20.847904 23035 raft_consensus.cc:697] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:20.848722 23035 consensus_queue.cc:237] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.863114 23019 consensus_queue.cc:1048] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.865971 22707 catalog_manager.cc:5697] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 0 to 1, leader changed from <none> to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:20.922847 23036 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.934170 23034 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:20.997041 23035 raft_consensus.cc:493] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.997599 23035 raft_consensus.cc:515] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:20.999351 23035 leader_election.cc:290] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.000309 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b" is_pre_election: true
I20260709 12:12:21.000380 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" is_pre_election: true
I20260709 12:12:21.000909 22884 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:21.001014 22958 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 0.
I20260709 12:12:21.002008 22775 leader_election.cc:304] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.002661 23035 raft_consensus.cc:2804] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:21.003011 23035 raft_consensus.cc:493] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.003302 23035 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:21.008739 23035 raft_consensus.cc:515] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.010543 23035 leader_election.cc:290] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.011386 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.011579 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.011863 22884 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:21.012136 22958 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:21.016842 22958 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:21.016875 22884 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 1.
I20260709 12:12:21.017987 22777 leader_election.cc:304] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:21.018913 23035 raft_consensus.cc:2804] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:21.019277 23035 raft_consensus.cc:697] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.020151 23035 consensus_queue.cc:237] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.026962 22707 catalog_manager.cc:5697] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 0 to 1, leader changed from <none> to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.077653 23035 consensus_queue.cc:1048] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.089380 23035 consensus_queue.cc:1048] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.109642 22884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:45686
I20260709 12:12:21.110381 22884 raft_consensus.cc:606] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:21.124995 22958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44570
I20260709 12:12:21.125634 22958 raft_consensus.cc:606] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:21.131232 22958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44570
I20260709 12:12:21.131872 22958 raft_consensus.cc:606] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:21.138736 22884 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:45686
I20260709 12:12:21.139369 22884 raft_consensus.cc:606] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:21.155751 23078 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.158881 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844a19913c634ce7b8fba9c809990942"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:41720
I20260709 12:12:21.159386 22811 raft_consensus.cc:606] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Received request to transfer leadership to TS e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:21.163250 23007 raft_consensus.cc:993] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.163647 23007 raft_consensus.cc:1081] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.165369 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44776
I20260709 12:12:21.165989 22811 raft_consensus.cc:493] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.166388 22811 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.167671 22958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
mode: GRACEFUL
new_leader_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:44570
I20260709 12:12:21.168267 22958 raft_consensus.cc:606] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Received request to transfer leadership to TS e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:21.170899 23007 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.174377 22811 raft_consensus.cc:515] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.175597 22810 raft_consensus.cc:1240] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Rejecting Update request from peer e5dd675310a7430aa685001cdd0ff16b for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:21.177911 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.178287 22811 leader_election.cc:290] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.178539 22884 raft_consensus.cc:3055] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.178897 22884 raft_consensus.cc:740] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:21.179723 22884 consensus_queue.cc:260] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.180752 22884 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.180647 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.186889 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:41720
I20260709 12:12:21.187676 22811 raft_consensus.cc:606] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Received request to transfer leadership to TS e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:21.190748 22884 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.192103 22775 leader_election.cc:304] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:21.192966 23035 raft_consensus.cc:2804] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.193535 23035 raft_consensus.cc:697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.193735 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:41720
I20260709 12:12:21.194350 22811 raft_consensus.cc:606] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Received request to transfer leadership to TS 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:21.194416 23035 consensus_queue.cc:237] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.201421 22709 catalog_manager.cc:5697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 1 to 2, leader changed from e5dd675310a7430aa685001cdd0ff16b (127.21.150.194) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.322711 23070 consensus_queue.cc:1048] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.329888 23034 raft_consensus.cc:993] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.330355 23034 raft_consensus.cc:1081] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.332650 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44788
I20260709 12:12:21.333238 22811 raft_consensus.cc:493] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.333655 22811 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.338673 23070 consensus_queue.cc:1048] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.346470 22811 raft_consensus.cc:515] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.348038 23008 raft_consensus.cc:993] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:21.348532 23070 raft_consensus.cc:1081] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Signalling peer 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:21.349668 22811 leader_election.cc:290] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.350615 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.351171 22884 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.352007 22958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:44564
I20260709 12:12:21.352501 22959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77a2992f247b4a41b3affc19472e2d36" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.352635 22958 raft_consensus.cc:493] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.353060 22958 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.353065 22959 raft_consensus.cc:3055] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.353755 22959 raft_consensus.cc:740] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:21.354581 22959 consensus_queue.cc:260] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.355999 22959 raft_consensus.cc:3060] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.358314 22884 raft_consensus.cc:2468] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.359830 22775 leader_election.cc:304] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.360656 23070 raft_consensus.cc:2804] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.361133 23070 raft_consensus.cc:697] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.361954 23070 consensus_queue.cc:237] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.362624 22959 raft_consensus.cc:2468] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.363354 22958 raft_consensus.cc:515] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.365239 23036 raft_consensus.cc:993] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.365684 23036 raft_consensus.cc:1081] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.366335 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:21.366962 22811 raft_consensus.cc:3055] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.366950 22884 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "864a8e326a9944a5a8fed670ce1a84ec" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.367429 22810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44788
I20260709 12:12:21.367537 22811 raft_consensus.cc:740] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.367830 22884 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.368136 22810 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.368548 22810 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.368676 22811 consensus_queue.cc:260] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.369966 22811 raft_consensus.cc:3060] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.371002 22707 catalog_manager.cc:5697] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 1 to 2, leader changed from 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.373549 22958 leader_election.cc:290] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 2 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:21.375399 22810 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.376281 22884 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 2.
I20260709 12:12:21.376936 22811 raft_consensus.cc:2468] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 2.
I20260709 12:12:21.378486 22926 leader_election.cc:304] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:21.378501 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.379212 22885 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.380966 23036 raft_consensus.cc:2804] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.381531 23036 raft_consensus.cc:697] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:21.382256 23036 consensus_queue.cc:237] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.384572 22885 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.385670 22810 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.385821 22775 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.386348 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.386723 23070 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.387006 22958 raft_consensus.cc:3055] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.387257 23070 raft_consensus.cc:697] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.387475 22958 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:21.388160 23070 consensus_queue.cc:237] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.388341 22958 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.389724 22958 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.389844 22707 catalog_manager.cc:5697] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 1 to 2, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 2 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:21.397202 22958 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.396418 22709 catalog_manager.cc:5697] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 1 to 2, leader changed from 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.465956 23070 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.477972 23033 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:21.479864 23006 raft_consensus.cc:993] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: : Instructing follower e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.480625 23006 raft_consensus.cc:1081] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Signalling peer e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.482384 22884 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:43396
I20260709 12:12:21.482992 22884 raft_consensus.cc:493] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.483542 22884 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.487771 23033 raft_consensus.cc:993] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.488271 23008 raft_consensus.cc:1081] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Signalling peer e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.489974 22885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "844a19913c634ce7b8fba9c809990942"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:45670
I20260709 12:12:21.491155 22885 raft_consensus.cc:493] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.491498 22885 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.492514 22884 raft_consensus.cc:515] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.494199 22884 leader_election.cc:290] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.495076 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:21.495398 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.495638 22810 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.496004 22958 raft_consensus.cc:3055] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.496338 22958 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:21.497165 22958 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.498183 22885 raft_consensus.cc:515] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.498476 22958 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.501101 22810 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.501369 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:21.501976 22811 raft_consensus.cc:3055] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.502324 22811 raft_consensus.cc:740] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.502839 22852 leader_election.cc:304] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.503113 22811 consensus_queue.cc:260] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.503800 23007 raft_consensus.cc:2804] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.504240 22811 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.504287 23007 raft_consensus.cc:697] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:21.505342 22958 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.505259 23007 consensus_queue.cc:237] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.506601 22885 leader_election.cc:290] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.507964 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.508518 22958 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.510789 22811 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.511996 22852 leader_election.cc:304] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.512835 23007 raft_consensus.cc:2804] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.513263 23007 raft_consensus.cc:697] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:21.513993 23007 consensus_queue.cc:237] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.514819 22958 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.514385 22709 catalog_manager.cc:5697] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 1 to 2, leader changed from 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195) to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 2 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.526556 22709 catalog_manager.cc:5697] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 1 to 2, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 2 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.579185 23019 raft_consensus.cc:993] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.579950 23019 raft_consensus.cc:1081] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:21.581840 22810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44776
I20260709 12:12:21.582463 22810 raft_consensus.cc:493] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.582852 22810 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.588140 22810 raft_consensus.cc:515] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.590184 22810 leader_election.cc:290] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.591236 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:21.591781 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.592015 22885 raft_consensus.cc:3055] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.592341 22958 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.592370 22885 raft_consensus.cc:740] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:21.593510 22885 consensus_queue.cc:260] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.594767 22885 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.599833 22958 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.601147 22777 leader_election.cc:304] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:21.601565 22885 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:21.602082 23008 raft_consensus.cc:2804] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.602586 23008 raft_consensus.cc:697] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.603343 23008 consensus_queue.cc:237] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.609267 22709 catalog_manager.cc:5697] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 1 to 2, leader changed from e5dd675310a7430aa685001cdd0ff16b (127.21.150.194) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.678261 22958 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.683679 22958 raft_consensus.cc:1275] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.685050 23033 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.687805 22885 raft_consensus.cc:1275] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.689289 23033 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.697757 23070 raft_consensus.cc:993] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.698230 23070 raft_consensus.cc:1081] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Signalling peer e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:21.702473 22885 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:45670
I20260709 12:12:21.703064 22885 raft_consensus.cc:493] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:21.703466 22885 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.713150 22885 raft_consensus.cc:515] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.715248 22885 leader_election.cc:290] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:21.716228 22810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:21.716598 22958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db4907f4b11a455c9ac45cf80d6fbe8c" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:21.716830 22810 raft_consensus.cc:3055] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:21.717149 22958 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.717185 22810 raft_consensus.cc:740] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:21.718089 22810 consensus_queue.cc:260] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.719012 22810 raft_consensus.cc:3060] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:21.722792 22958 raft_consensus.cc:2468] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.723675 22810 raft_consensus.cc:2468] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 2.
I20260709 12:12:21.723847 22852 leader_election.cc:304] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:21.724624 23007 raft_consensus.cc:2804] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:21.725036 23007 raft_consensus.cc:697] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:21.725785 23007 consensus_queue.cc:237] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:21.732384 22709 catalog_manager.cc:5697] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 1 to 2, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 2 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:21.763616 22958 raft_consensus.cc:1275] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.765005 23091 consensus_queue.cc:1048] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.774405 22885 raft_consensus.cc:1275] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.776257 23070 consensus_queue.cc:1048] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.789847 22958 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.791832 23070 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.797097 22810 raft_consensus.cc:1275] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.799382 23036 consensus_queue.cc:1048] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.801563 22885 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.803526 23091 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.810060 22886 raft_consensus.cc:1275] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.811901 23092 consensus_queue.cc:1048] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.983948 22885 raft_consensus.cc:1275] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.986021 23091 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.990126 22958 raft_consensus.cc:1275] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.991873 23091 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:21.992758 22958 raft_consensus.cc:1275] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.996182 22958 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:21.997430 23019 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:21.998077 23078 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:22.013612 22810 raft_consensus.cc:1275] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:22.015244 23078 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:22.020224 22811 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:22.025696 23096 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:22.198397 22811 raft_consensus.cc:1275] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:22.199692 23096 consensus_queue.cc:1048] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:22.210063 22957 raft_consensus.cc:1275] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:22.211771 23007 consensus_queue.cc:1048] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:24.233857 22107 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:12:24.234550 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:24.252230 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:24.252892 22811 raft_consensus.cc:606] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Received request to transfer leadership to TS 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:24.255800 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:24.256417 22811 raft_consensus.cc:606] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Received request to transfer leadership to TS 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:24.257791 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:12:24.258533 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:24.350596 23033 raft_consensus.cc:993] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:24.351048 23033 raft_consensus.cc:1081] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Signalling peer 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:24.352561 22957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:44564
I20260709 12:12:24.353000 22957 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.353289 22957 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.359373 22957 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.361683 22957 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 2 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:24.362337 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:24.362922 22811 raft_consensus.cc:3055] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:24.363330 22811 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:24.363008 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:24.363901 22885 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.364112 22811 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.365478 22811 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:24.369328 22885 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 2.
I20260709 12:12:24.370733 22924 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:24.371130 22811 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 2.
I20260709 12:12:24.371497 23100 raft_consensus.cc:2804] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:24.372633 23100 raft_consensus.cc:697] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:24.373518 23100 consensus_queue.cc:237] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.380540 22708 catalog_manager.cc:5697] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 1 to 2, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 2 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:24.658859 23104 raft_consensus.cc:993] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:24.659438 23033 raft_consensus.cc:1081] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Signalling peer 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:24.661310 22957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:44564
I20260709 12:12:24.661993 22957 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:24.662387 22957 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:24.702440 22957 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.704818 22957 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 3 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:24.706665 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:24.707629 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:24.707998 22811 raft_consensus.cc:3055] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:24.708420 22811 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:24.708437 22885 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:24.709755 22811 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.711452 22811 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:24.715503 22885 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 3.
I20260709 12:12:24.717051 22924 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:24.717561 22811 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 3.
I20260709 12:12:24.718092 23100 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:24.718660 23100 raft_consensus.cc:697] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:24.720005 23100 consensus_queue.cc:237] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:24.734987 22708 catalog_manager.cc:5697] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 2 to 3, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 3 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:24.904619 22885 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:24.906188 23100 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:24.920051 22811 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:24.922350 23105 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:25.163223 22811 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:25.164880 23115 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:25.175567 22885 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:25.177502 23115 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:25.260335 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:25.266301 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:25.266793 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:26.267915 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:26.273483 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:26.273952 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:27.274942 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.280750 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:27.281251 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:28.282405 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:28.289390 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:28.289929 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:29.290908 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:29.296908 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:29.297477 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:30.298424 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:30.303591 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:30.303995 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:31.304997 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:31.310721 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:31.311185 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:32.312165 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:32.317704 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:32.318135 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:33.319149 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:33.325719 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:33.326224 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:34.356465 22107 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:12:34.363511 22957 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44570
I20260709 12:12:34.364274 22957 raft_consensus.cc:606] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:34.373791 22957 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44570
I20260709 12:12:34.374249 22957 raft_consensus.cc:606] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:34.378611 22886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "844a19913c634ce7b8fba9c809990942"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:45686
I20260709 12:12:34.379084 22886 raft_consensus.cc:606] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:34.384295 22886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:45686
I20260709 12:12:34.384771 22886 raft_consensus.cc:606] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:34.389464 22886 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
mode: GRACEFUL
new_leader_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:45686
I20260709 12:12:34.389952 22886 raft_consensus.cc:606] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Received request to transfer leadership to TS 872c1e6c20d44893bb11247b8ba40760
I20260709 12:12:34.628294 23128 raft_consensus.cc:993] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.628968 23124 raft_consensus.cc:1081] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.630784 22810 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44776
I20260709 12:12:34.631395 22810 raft_consensus.cc:493] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.631785 22810 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.632817 23129 raft_consensus.cc:993] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.633275 23127 raft_consensus.cc:1081] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.634994 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44788
I20260709 12:12:34.635459 22811 raft_consensus.cc:493] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.635747 22811 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:34.638290 22810 raft_consensus.cc:515] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.640156 22811 raft_consensus.cc:515] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.640901 22810 leader_election.cc:290] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:34.641929 22811 leader_election.cc:290] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 4 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:34.641865 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:34.642670 22957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4badfdfaa1b04d92a3ace654c6a7397c" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:34.643067 22886 raft_consensus.cc:3055] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:34.643333 22957 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.643450 22886 raft_consensus.cc:740] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:34.643231 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:34.642760 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5ea0116fc44819c4e341f9ec4069b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:34.644062 22960 raft_consensus.cc:3055] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:34.644263 22885 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:34.644423 22960 raft_consensus.cc:740] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:34.644506 22886 consensus_queue.cc:260] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.645367 22960 consensus_queue.cc:260] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.646090 22886 raft_consensus.cc:3060] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:34.646535 22960 raft_consensus.cc:3060] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:34.650045 22957 raft_consensus.cc:2468] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:34.651134 22885 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 4.
I20260709 12:12:34.651558 22777 leader_election.cc:304] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:34.652601 22775 leader_election.cc:304] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:34.653162 22886 raft_consensus.cc:2468] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 2.
I20260709 12:12:34.653189 23121 raft_consensus.cc:2804] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:34.653888 22960 raft_consensus.cc:2468] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 4.
I20260709 12:12:34.654570 23130 raft_consensus.cc:2804] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:34.655927 23121 raft_consensus.cc:697] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:34.657003 23130 raft_consensus.cc:697] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 4 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:34.656862 23121 consensus_queue.cc:237] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.657810 23130 consensus_queue.cc:237] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.665422 22708 catalog_manager.cc:5697] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 1 to 2, leader changed from e5dd675310a7430aa685001cdd0ff16b (127.21.150.194) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:34.666620 22708 catalog_manager.cc:5697] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 3 to 4, leader changed from 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:34.736194 23128 raft_consensus.cc:993] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.737337 23124 raft_consensus.cc:1081] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.739449 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "844a19913c634ce7b8fba9c809990942"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44776
I20260709 12:12:34.740044 22811 raft_consensus.cc:493] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.740370 22811 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.747133 22811 raft_consensus.cc:515] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.749065 22811 leader_election.cc:290] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:34.751331 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:34.751331 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "844a19913c634ce7b8fba9c809990942" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:34.752395 22885 raft_consensus.cc:3055] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:34.752564 22960 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.752784 22885 raft_consensus.cc:740] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:34.753954 22885 consensus_queue.cc:260] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.755668 22885 raft_consensus.cc:3060] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.761047 22960 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.762472 22777 leader_election.cc:304] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:34.762714 22885 raft_consensus.cc:2468] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.763439 23130 raft_consensus.cc:2804] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:34.763878 23130 raft_consensus.cc:697] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:34.764837 23130 consensus_queue.cc:237] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.774364 22707 catalog_manager.cc:5697] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 2 to 3, leader changed from e5dd675310a7430aa685001cdd0ff16b (127.21.150.194) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:34.797945 23128 raft_consensus.cc:993] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.798425 23124 raft_consensus.cc:1081] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.799813 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44776
I20260709 12:12:34.800256 22811 raft_consensus.cc:493] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.800532 22811 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.805140 22811 raft_consensus.cc:515] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.806814 22811 leader_election.cc:290] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:34.807740 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:34.807930 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:34.808259 22885 raft_consensus.cc:3055] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:34.808449 22960 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.808559 22885 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:34.809405 22885 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.810314 22885 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.813707 22960 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.814826 22885 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.814918 22777 leader_election.cc:304] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:34.815831 23130 raft_consensus.cc:2804] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:34.816260 23130 raft_consensus.cc:697] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:34.817080 23130 consensus_queue.cc:237] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.824072 22707 catalog_manager.cc:5697] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 2 to 3, leader changed from e5dd675310a7430aa685001cdd0ff16b (127.21.150.194) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:34.877032 23129 raft_consensus.cc:993] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: : Instructing follower 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.877890 23127 raft_consensus.cc:1081] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Signalling peer 872c1e6c20d44893bb11247b8ba40760 to start an election
I20260709 12:12:34.879405 22811 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
 from {username='slave'} at 127.0.0.1:44788
I20260709 12:12:34.879868 22811 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:34.880164 22811 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.884272 22811 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.885798 22811 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Requested vote from peers e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:34.886601 22885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:34.886917 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "872c1e6c20d44893bb11247b8ba40760" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:34.887173 22885 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.887437 22960 raft_consensus.cc:3055] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:34.887758 22960 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:34.888482 22960 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.889688 22960 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:34.891839 22885 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.892993 22775 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 872c1e6c20d44893bb11247b8ba40760, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:34.893888 23130 raft_consensus.cc:2804] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:34.894332 23130 raft_consensus.cc:697] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Becoming Leader. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:34.895219 22960 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 872c1e6c20d44893bb11247b8ba40760 in term 3.
I20260709 12:12:34.895112 23130 consensus_queue.cc:237] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:34.902616 22707 catalog_manager.cc:5697] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 reported cstate change: term changed from 2 to 3, leader changed from 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195) to 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "872c1e6c20d44893bb11247b8ba40760" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:35.037264 22960 raft_consensus.cc:1275] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:35.038885 23130 consensus_queue.cc:1048] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:35.050220 22885 raft_consensus.cc:1275] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:35.051786 23132 consensus_queue.cc:1048] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:12:35.145236 22960 raft_consensus.cc:1275] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.146687 23132 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.152813 22885 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:35.154313 23132 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:35.158514 22885 raft_consensus.cc:1275] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.160905 23132 consensus_queue.cc:1048] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.164942 22960 raft_consensus.cc:1275] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:35.166630 23121 consensus_queue.cc:1048] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:12:35.219404 22960 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.220623 23132 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.229375 22885 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.230710 23132 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.309883 22885 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.310974 23132 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:35.319170 22960 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Refusing update from remote peer 872c1e6c20d44893bb11247b8ba40760: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:35.320989 23121 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:37.427878 22107 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:12:37.428850 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:37.437000 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:37.437551 22811 raft_consensus.cc:606] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Received request to transfer leadership to TS e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:37.439662 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:37.440053 22811 raft_consensus.cc:606] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Received request to transfer leadership to TS 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:37.442003 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:37.442406 22811 raft_consensus.cc:606] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Received request to transfer leadership to TS e5dd675310a7430aa685001cdd0ff16b
I20260709 12:12:37.444319 22811 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
mode: GRACEFUL
new_leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:41724
I20260709 12:12:37.444654 22811 raft_consensus.cc:606] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Received request to transfer leadership to TS 2d2b7ff1e3c5440fa3989dc0455bd77a
I20260709 12:12:37.445448 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:12:37.445883 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:37.495798 23158 raft_consensus.cc:993] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:37.496483 23121 raft_consensus.cc:1081] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Signalling peer e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:37.497910 22886 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:45670
I20260709 12:12:37.498317 22886 raft_consensus.cc:493] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.498570 22886 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.503167 22886 raft_consensus.cc:515] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.505076 22886 leader_election.cc:290] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 4 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:37.505681 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:37.506137 22957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6216945e97b14cffba0ece229f0ed12b" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:37.506239 22811 raft_consensus.cc:3055] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:37.506608 22957 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.506637 22811 raft_consensus.cc:740] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:37.507391 22811 consensus_queue.cc:260] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.508244 22811 raft_consensus.cc:3060] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.511405 22957 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 4.
I20260709 12:12:37.512485 22852 leader_election.cc:304] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:37.512642 22811 raft_consensus.cc:2468] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 4.
I20260709 12:12:37.513150 23157 raft_consensus.cc:2804] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:37.514282 23157 raft_consensus.cc:697] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 4 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:37.515120 23157 consensus_queue.cc:237] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.521040 22707 catalog_manager.cc:5697] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 3 to 4, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 4 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:37.534498 23158 raft_consensus.cc:993] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:37.535652 23121 raft_consensus.cc:1081] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Signalling peer 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:37.537379 22957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:44564
I20260709 12:12:37.537835 22957 raft_consensus.cc:493] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.538185 22957 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.542444 22957 raft_consensus.cc:515] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.545017 22957 leader_election.cc:290] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 4 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:37.545944 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:37.545799 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725faa936c0c4ac8ac9696dfaa1470de" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:37.546687 22811 raft_consensus.cc:3055] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:37.546700 22886 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.547206 22811 raft_consensus.cc:740] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:37.547930 22811 consensus_queue.cc:260] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.548780 22811 raft_consensus.cc:3060] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:37.551539 22886 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 4.
I20260709 12:12:37.552724 22924 leader_election.cc:304] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:37.553171 22811 raft_consensus.cc:2468] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 4.
I20260709 12:12:37.553581 23127 raft_consensus.cc:2804] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:37.554745 23127 raft_consensus.cc:697] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:37.555522 23127 consensus_queue.cc:237] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.562282 22707 catalog_manager.cc:5697] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 3 to 4, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 4 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:37.890448 23158 raft_consensus.cc:993] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:37.890954 23121 raft_consensus.cc:1081] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Signalling peer e5dd675310a7430aa685001cdd0ff16b to start an election
I20260709 12:12:37.892532 22886 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378"
dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
 from {username='slave'} at 127.0.0.1:45670
I20260709 12:12:37.892930 22886 raft_consensus.cc:493] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:37.893198 22886 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.897411 22886 raft_consensus.cc:515] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.898957 22886 leader_election.cc:290] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 3 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195:42245)
I20260709 12:12:37.899660 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:37.899864 22960 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca67b03f1ef4e4fbf649384cb9d8378" candidate_uuid: "e5dd675310a7430aa685001cdd0ff16b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
I20260709 12:12:37.900233 22811 raft_consensus.cc:3055] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:37.900364 22960 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.900564 22811 raft_consensus.cc:740] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:37.901404 22811 consensus_queue.cc:260] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.902688 22811 raft_consensus.cc:3060] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:37.907676 22960 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 3.
I20260709 12:12:37.908947 22852 leader_election.cc:304] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, e5dd675310a7430aa685001cdd0ff16b; no voters: 
I20260709 12:12:37.909606 22811 raft_consensus.cc:2468] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5dd675310a7430aa685001cdd0ff16b in term 3.
I20260709 12:12:37.909758 23157 raft_consensus.cc:2804] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:37.910310 23157 raft_consensus.cc:697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 3 LEADER]: Becoming Leader. State: Replica: e5dd675310a7430aa685001cdd0ff16b, State: Running, Role: LEADER
I20260709 12:12:37.911150 23157 consensus_queue.cc:237] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:37.918947 22707 catalog_manager.cc:5697] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b reported cstate change: term changed from 2 to 3, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to e5dd675310a7430aa685001cdd0ff16b (127.21.150.194). New cstate: current_term: 3 leader_uuid: "e5dd675310a7430aa685001cdd0ff16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:37.968974 22811 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:37.970180 23127 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:37.980154 22886 raft_consensus.cc:1275] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:37.981544 23127 consensus_queue.cc:1048] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:38.022303 22811 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:38.023383 23157 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:38.027549 23121 raft_consensus.cc:993] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: : Instructing follower 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:38.029222 23121 raft_consensus.cc:1081] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Signalling peer 2d2b7ff1e3c5440fa3989dc0455bd77a to start an election
I20260709 12:12:38.031013 22957 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75"
dest_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a"
 from {username='slave'} at 127.0.0.1:44564
I20260709 12:12:38.031615 22957 raft_consensus.cc:493] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:38.032011 22957 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.037346 22960 raft_consensus.cc:1275] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:12:38.038915 22957 raft_consensus.cc:515] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:38.039213 23157 consensus_queue.cc:1048] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:12:38.041903 22957 leader_election.cc:290] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 3 election: Requested vote from peers 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155), e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991)
I20260709 12:12:38.041818 22886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e5dd675310a7430aa685001cdd0ff16b"
I20260709 12:12:38.042449 22886 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.041471 22811 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4afe3dc6a22149639feac8d506b61d75" candidate_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "872c1e6c20d44893bb11247b8ba40760"
I20260709 12:12:38.043632 22811 raft_consensus.cc:3055] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:12:38.044019 22811 raft_consensus.cc:740] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 872c1e6c20d44893bb11247b8ba40760, State: Running, Role: LEADER
I20260709 12:12:38.044958 22811 consensus_queue.cc:260] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:38.046701 22811 raft_consensus.cc:3060] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:38.052908 22811 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 3.
I20260709 12:12:38.052848 22886 raft_consensus.cc:2468] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2b7ff1e3c5440fa3989dc0455bd77a in term 3.
I20260709 12:12:38.054255 22926 leader_election.cc:304] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2b7ff1e3c5440fa3989dc0455bd77a, 872c1e6c20d44893bb11247b8ba40760; no voters: 
I20260709 12:12:38.055660 23172 raft_consensus.cc:2804] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:38.056205 23172 raft_consensus.cc:697] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Becoming Leader. State: Replica: 2d2b7ff1e3c5440fa3989dc0455bd77a, State: Running, Role: LEADER
I20260709 12:12:38.057145 23172 consensus_queue.cc:237] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } }
I20260709 12:12:38.065943 22707 catalog_manager.cc:5697] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a reported cstate change: term changed from 2 to 3, leader changed from 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193) to 2d2b7ff1e3c5440fa3989dc0455bd77a (127.21.150.195). New cstate: current_term: 3 leader_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:38.292397 22811 raft_consensus.cc:1275] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:38.293587 23157 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:38.302037 22957 raft_consensus.cc:1275] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Refusing update from remote peer e5dd675310a7430aa685001cdd0ff16b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:38.303947 23176 consensus_queue.cc:1048] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2b7ff1e3c5440fa3989dc0455bd77a" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 42245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:38.443233 22811 raft_consensus.cc:1275] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:38.444499 23172 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "872c1e6c20d44893bb11247b8ba40760" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 33155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:38.446966 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:38.453316 22886 raft_consensus.cc:1275] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Refusing update from remote peer 2d2b7ff1e3c5440fa3989dc0455bd77a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:12:38.454654 23166 consensus_queue.cc:1048] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5dd675310a7430aa685001cdd0ff16b" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:12:38.459218 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:38.460935 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:39.462106 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:39.468765 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:39.469211 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:40.470269 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:40.475699 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:40.476153 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:41.477003 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:41.482321 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:41.482774 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:42.483690 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:42.489251 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:42.489691 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:43.490675 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:43.496248 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:43.496683 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:44.497789 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:44.504134 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:44.504844 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:45.506354 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:45.513509 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:45.513970 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:46.515040 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:46.520866 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:12:46.521338 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:47.556270 22107 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:12:47.557135 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
W20260709 12:12:47.571995 22852 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 30 similar messages]
W20260709 12:12:47.577092 22852 consensus_peers.cc:597] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b -> Peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Couldn't send request to peer 872c1e6c20d44893bb11247b8ba40760. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.590832 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:47.591538 22107 tablet_replica.cc:333] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.592137 22107 raft_consensus.cc:2243] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.593015 22107 raft_consensus.cc:2272] T 844a19913c634ce7b8fba9c809990942 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.594990 22107 tablet_replica.cc:333] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.595487 22107 raft_consensus.cc:2243] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.595897 22107 raft_consensus.cc:2272] T 864a8e326a9944a5a8fed670ce1a84ec P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.598309 22107 tablet_replica.cc:333] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.599045 22107 raft_consensus.cc:2243] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.599869 22107 raft_consensus.cc:2272] T 4badfdfaa1b04d92a3ace654c6a7397c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.601742 22107 tablet_replica.cc:333] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.602262 22107 raft_consensus.cc:2243] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.602730 22107 raft_consensus.cc:2272] T db4907f4b11a455c9ac45cf80d6fbe8c P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.604522 22107 tablet_replica.cc:333] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.605104 22107 raft_consensus.cc:2243] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.605630 22107 raft_consensus.cc:2272] T 6216945e97b14cffba0ece229f0ed12b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.607443 22107 tablet_replica.cc:333] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.607959 22107 raft_consensus.cc:2243] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.608402 22107 raft_consensus.cc:2272] T 725faa936c0c4ac8ac9696dfaa1470de P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.610255 22107 tablet_replica.cc:333] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.610744 22107 raft_consensus.cc:2243] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.611196 22107 raft_consensus.cc:2272] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.612931 22107 tablet_replica.cc:333] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.613402 22107 raft_consensus.cc:2243] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.614254 22107 raft_consensus.cc:2272] T 42a5ea0116fc44819c4e341f9ec4069b P 872c1e6c20d44893bb11247b8ba40760 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.616046 22107 tablet_replica.cc:333] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.616505 22107 raft_consensus.cc:2243] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.617246 22107 raft_consensus.cc:2272] T 77a2992f247b4a41b3affc19472e2d36 P 872c1e6c20d44893bb11247b8ba40760 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.618901 22107 tablet_replica.cc:333] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760: stopping tablet replica
I20260709 12:12:47.619339 22107 raft_consensus.cc:2243] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.619757 22107 raft_consensus.cc:2272] T 4afe3dc6a22149639feac8d506b61d75 P 872c1e6c20d44893bb11247b8ba40760 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.652261 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
W20260709 12:12:47.675235 22926 consensus_peers.cc:597] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a -> Peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Couldn't send request to peer 872c1e6c20d44893bb11247b8ba40760. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:33155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.677947 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:12:47.696120 22926 consensus_peers.cc:597] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a -> Peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Couldn't send request to peer 872c1e6c20d44893bb11247b8ba40760. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:33155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:47.706581 22852 consensus_peers.cc:597] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b -> Peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Couldn't send request to peer 872c1e6c20d44893bb11247b8ba40760. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:33155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.709673 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:47.710436 22107 tablet_replica.cc:333] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.711181 22107 raft_consensus.cc:2243] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.712003 22107 raft_consensus.cc:2272] T db4907f4b11a455c9ac45cf80d6fbe8c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.714136 22107 tablet_replica.cc:333] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.714712 22107 raft_consensus.cc:2243] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.715512 22107 raft_consensus.cc:2272] T 6216945e97b14cffba0ece229f0ed12b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.717468 22107 tablet_replica.cc:333] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.718026 22107 raft_consensus.cc:2243] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.718474 22107 raft_consensus.cc:2272] T 864a8e326a9944a5a8fed670ce1a84ec P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.720216 22107 tablet_replica.cc:333] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.720731 22107 raft_consensus.cc:2243] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.721179 22107 raft_consensus.cc:2272] T 4badfdfaa1b04d92a3ace654c6a7397c P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.723080 22107 tablet_replica.cc:333] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.723613 22107 raft_consensus.cc:2243] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.723984 22107 raft_consensus.cc:2272] T 844a19913c634ce7b8fba9c809990942 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.725863 22107 tablet_replica.cc:333] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.726397 22107 raft_consensus.cc:2243] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.727222 22107 raft_consensus.cc:2272] T 1ca67b03f1ef4e4fbf649384cb9d8378 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.728921 22107 tablet_replica.cc:333] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.729419 22107 raft_consensus.cc:2243] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.729944 22107 raft_consensus.cc:2272] T 725faa936c0c4ac8ac9696dfaa1470de P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.731812 22107 tablet_replica.cc:333] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.732290 22107 raft_consensus.cc:2243] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.732758 22107 raft_consensus.cc:2272] T 42a5ea0116fc44819c4e341f9ec4069b P e5dd675310a7430aa685001cdd0ff16b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.734452 22107 tablet_replica.cc:333] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.734911 22107 raft_consensus.cc:2243] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.735327 22107 raft_consensus.cc:2272] T 77a2992f247b4a41b3affc19472e2d36 P e5dd675310a7430aa685001cdd0ff16b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.737016 22107 tablet_replica.cc:333] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b: stopping tablet replica
I20260709 12:12:47.737461 22107 raft_consensus.cc:2243] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.737883 22107 raft_consensus.cc:2272] T 4afe3dc6a22149639feac8d506b61d75 P e5dd675310a7430aa685001cdd0ff16b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:12:47.758854 22926 consensus_peers.cc:597] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a -> Peer 872c1e6c20d44893bb11247b8ba40760 (127.21.150.193:33155): Couldn't send request to peer 872c1e6c20d44893bb11247b8ba40760. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:33155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.768853 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
W20260709 12:12:47.770488 22924 consensus_peers.cc:597] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a -> Peer e5dd675310a7430aa685001cdd0ff16b (127.21.150.194:40991): Couldn't send request to peer e5dd675310a7430aa685001cdd0ff16b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:40991: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:47.790242 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:12:47.817046 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:12:47.817857 22107 tablet_replica.cc:333] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.818517 22107 raft_consensus.cc:2243] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.819051 22107 raft_consensus.cc:2272] T 844a19913c634ce7b8fba9c809990942 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.820935 22107 tablet_replica.cc:333] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.821529 22107 raft_consensus.cc:2243] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.822297 22107 raft_consensus.cc:2272] T 4afe3dc6a22149639feac8d506b61d75 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.824167 22107 tablet_replica.cc:333] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.824683 22107 raft_consensus.cc:2243] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.825145 22107 raft_consensus.cc:2272] T 1ca67b03f1ef4e4fbf649384cb9d8378 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.827138 22107 tablet_replica.cc:333] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.827643 22107 raft_consensus.cc:2243] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.828387 22107 raft_consensus.cc:2272] T 725faa936c0c4ac8ac9696dfaa1470de P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.830078 22107 tablet_replica.cc:333] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.830570 22107 raft_consensus.cc:2243] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.831079 22107 raft_consensus.cc:2272] T db4907f4b11a455c9ac45cf80d6fbe8c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.832726 22107 tablet_replica.cc:333] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.833209 22107 raft_consensus.cc:2243] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.833660 22107 raft_consensus.cc:2272] T 6216945e97b14cffba0ece229f0ed12b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.835229 22107 tablet_replica.cc:333] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.835700 22107 raft_consensus.cc:2243] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.836716 22107 raft_consensus.cc:2272] T 864a8e326a9944a5a8fed670ce1a84ec P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.839236 22107 tablet_replica.cc:333] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.839972 22107 raft_consensus.cc:2243] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.840487 22107 raft_consensus.cc:2272] T 4badfdfaa1b04d92a3ace654c6a7397c P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.842685 22107 tablet_replica.cc:333] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.843189 22107 raft_consensus.cc:2243] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.843628 22107 raft_consensus.cc:2272] T 77a2992f247b4a41b3affc19472e2d36 P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.845394 22107 tablet_replica.cc:333] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a: stopping tablet replica
I20260709 12:12:47.845927 22107 raft_consensus.cc:2243] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:12:47.846360 22107 raft_consensus.cc:2272] T 42a5ea0116fc44819c4e341f9ec4069b P 2d2b7ff1e3c5440fa3989dc0455bd77a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.876291 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:12:47.897063 22107 master.cc:562] Master@127.21.150.254:39721 shutting down...
I20260709 12:12:47.920655 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:12:47.921418 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:12:47.921838 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 00abdabf4916465083aeda8a2667460e: stopping tablet replica
I20260709 12:12:47.941897 22107 master.cc:584] Master@127.21.150.254:39721 shutdown complete.
2d2b7ff1e3c5440fa3989dc0455bd77a  1
872c1e6c20d44893bb11247b8ba40760  5
e5dd675310a7430aa685001cdd0ff16b  4
2d2b7ff1e3c5440fa3989dc0455bd77a  3
872c1e6c20d44893bb11247b8ba40760  3
e5dd675310a7430aa685001cdd0ff16b  4
2d2b7ff1e3c5440fa3989dc0455bd77a  1
872c1e6c20d44893bb11247b8ba40760  8
e5dd675310a7430aa685001cdd0ff16b  1
2d2b7ff1e3c5440fa3989dc0455bd77a  3
872c1e6c20d44893bb11247b8ba40760  4
e5dd675310a7430aa685001cdd0ff16b  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28991 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:12:47.985868 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:44861
I20260709 12:12:47.986937 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:47.993603 23195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:47.993675 23194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:47.994510 23197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:47.994954 22107 server_base.cc:1061] running on GCE node
I20260709 12:12:47.995904 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:47.996124 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:47.996320 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599167996278 us; error 0 us; skew 500 ppm
I20260709 12:12:47.996842 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:48.000361 22107 webserver.cc:533] Webserver started at http://127.21.150.254:40175/ using document root <none> and password file <none>
I20260709 12:12:48.000873 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:48.001092 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:48.001394 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:48.002602 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "259b912070e34dfba6c09de6bbb3d141"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.007148 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:12:48.010514 23202 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.011288 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:12:48.011587 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "259b912070e34dfba6c09de6bbb3d141"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.011899 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:48.027812 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:48.029103 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:48.067375 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:44861
I20260709 12:12:48.067432 23253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:44861 every 8 connection(s)
I20260709 12:12:48.071457 23254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.081350 23254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Bootstrap starting.
I20260709 12:12:48.086025 23254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:48.090127 23254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: No bootstrap required, opened a new log
I20260709 12:12:48.092213 23254 raft_consensus.cc:359] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER }
I20260709 12:12:48.092613 23254 raft_consensus.cc:385] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:48.092796 23254 raft_consensus.cc:740] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259b912070e34dfba6c09de6bbb3d141, State: Initialized, Role: FOLLOWER
I20260709 12:12:48.093258 23254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER }
I20260709 12:12:48.093761 23254 raft_consensus.cc:399] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:12:48.093954 23254 raft_consensus.cc:493] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:12:48.094164 23254 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:48.098191 23254 raft_consensus.cc:515] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER }
I20260709 12:12:48.098649 23254 leader_election.cc:304] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 259b912070e34dfba6c09de6bbb3d141; no voters: 
I20260709 12:12:48.099723 23254 leader_election.cc:290] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:12:48.100158 23257 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:48.101495 23257 raft_consensus.cc:697] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 1 LEADER]: Becoming Leader. State: Replica: 259b912070e34dfba6c09de6bbb3d141, State: Running, Role: LEADER
I20260709 12:12:48.102257 23257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER }
I20260709 12:12:48.102762 23254 sys_catalog.cc:565] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:12:48.107712 23258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "259b912070e34dfba6c09de6bbb3d141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER } }
I20260709 12:12:48.107594 23259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 259b912070e34dfba6c09de6bbb3d141. Latest consensus state: current_term: 1 leader_uuid: "259b912070e34dfba6c09de6bbb3d141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259b912070e34dfba6c09de6bbb3d141" member_type: VOTER } }
I20260709 12:12:48.108316 23259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:48.109952 23258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [sys.catalog]: This master's current role is: LEADER
I20260709 12:12:48.110440 23266 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:12:48.116706 23266 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:12:48.117727 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:12:48.126259 23266 catalog_manager.cc:1383] Generated new cluster ID: 5f4080b258dc4142a6f511b9c89c3876
I20260709 12:12:48.126561 23266 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:12:48.163892 23266 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:12:48.165141 23266 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:12:48.177353 23266 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Generated new TSK 0
I20260709 12:12:48.178032 23266 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:12:48.184206 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:48.190155 23275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:48.191654 23276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.195425 22107 server_base.cc:1061] running on GCE node
W20260709 12:12:48.195858 23278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.196966 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:48.197175 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:48.197364 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599168197344 us; error 0 us; skew 500 ppm
I20260709 12:12:48.197979 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:48.200234 22107 webserver.cc:533] Webserver started at http://127.21.150.193:42925/ using document root <none> and password file <none>
I20260709 12:12:48.200729 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:48.200934 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:48.201220 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:48.202366 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "2778b449761a46708f2f643b1b5b5854"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.206918 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:12:48.210810 23283 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.211774 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:12:48.212100 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "2778b449761a46708f2f643b1b5b5854"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.212438 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:48.231693 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:48.232900 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:48.234370 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:48.236655 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:48.236876 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.237123 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:48.237319 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.279086 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:46055
I20260709 12:12:48.279204 23345 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:46055 every 8 connection(s)
I20260709 12:12:48.283998 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:48.291388 23350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:48.293772 23351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.294898 22107 server_base.cc:1061] running on GCE node
W20260709 12:12:48.295152 23353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.296967 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:48.297158 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:48.297389 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599168297368 us; error 0 us; skew 500 ppm
I20260709 12:12:48.297713 23346 heartbeater.cc:344] Connected to a master server at 127.21.150.254:44861
I20260709 12:12:48.298049 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:48.298077 23346 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:48.298987 23346 heartbeater.cc:507] Master 127.21.150.254:44861 requested a full tablet report, sending...
I20260709 12:12:48.301604 22107 webserver.cc:533] Webserver started at http://127.21.150.194:44415/ using document root <none> and password file <none>
I20260709 12:12:48.301668 23219 ts_manager.cc:194] Registered new tserver with Master: 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:12:48.302187 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:48.302431 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:48.302706 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:48.303618 23219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44670
I20260709 12:12:48.303951 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "6253b834ffb744579b46fcede2fc0e91"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.308497 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:12:48.311697 23358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.312414 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:12:48.312701 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "6253b834ffb744579b46fcede2fc0e91"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.312981 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:48.331344 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:48.332508 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:48.333937 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:48.336117 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:48.336339 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.336576 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:48.337028 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.376242 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:41721
I20260709 12:12:48.376391 23420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:41721 every 8 connection(s)
I20260709 12:12:48.380918 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:48.387770 23424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:48.389382 23425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.390815 22107 server_base.cc:1061] running on GCE node
W20260709 12:12:48.392148 23427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:48.393239 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:48.393553 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:48.393779 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599168393760 us; error 0 us; skew 500 ppm
I20260709 12:12:48.394533 23421 heartbeater.cc:344] Connected to a master server at 127.21.150.254:44861
I20260709 12:12:48.394611 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:48.394914 23421 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:48.395591 23421 heartbeater.cc:507] Master 127.21.150.254:44861 requested a full tablet report, sending...
I20260709 12:12:48.397559 22107 webserver.cc:533] Webserver started at http://127.21.150.195:33605/ using document root <none> and password file <none>
I20260709 12:12:48.397657 23219 ts_manager.cc:194] Registered new tserver with Master: 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:48.398111 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:48.398350 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:48.398612 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:48.399145 23219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44674
I20260709 12:12:48.399799 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "da1e2fd157894634bf92d6c82c06ec64"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.404227 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:12:48.407476 23432 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.408226 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:12:48.408509 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "da1e2fd157894634bf92d6c82c06ec64"
format_stamp: "Formatted at 2026-07-09 12:12:48 on dist-test-slave-zctp"
I20260709 12:12:48.408793 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:48.431391 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:48.432587 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:48.434087 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:48.436228 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:48.436470 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.436722 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:48.436903 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:48.476176 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:33909
I20260709 12:12:48.476291 23494 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:33909 every 8 connection(s)
I20260709 12:12:48.489938 23495 heartbeater.cc:344] Connected to a master server at 127.21.150.254:44861
I20260709 12:12:48.490267 23495 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:48.490973 23495 heartbeater.cc:507] Master 127.21.150.254:44861 requested a full tablet report, sending...
I20260709 12:12:48.492983 23219 ts_manager.cc:194] Registered new tserver with Master: da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:48.493947 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01448389s
I20260709 12:12:48.494369 23219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44678
I20260709 12:12:48.519404 23219 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44680:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:12:48.525362 23219 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:12:48.721266 23383 tablet_service.cc:1511] Processing CreateTablet for tablet f55bd3b311de43ba8aca6be22937f68a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:48.722916 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55bd3b311de43ba8aca6be22937f68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.722564 23386 tablet_service.cc:1511] Processing CreateTablet for tablet 1bca0d75a0cb485ebd79c619c3bf8341 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:48.723872 23386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bca0d75a0cb485ebd79c619c3bf8341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.726599 23385 tablet_service.cc:1511] Processing CreateTablet for tablet 19d9facb5378417aaa3279b3dd9f02a9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:48.726889 23382 tablet_service.cc:1511] Processing CreateTablet for tablet 709006b08c044ed38e2df81795ed1613 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:48.728101 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d9facb5378417aaa3279b3dd9f02a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.721266 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:48.729696 23381 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:48.731766 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 1bca0d75a0cb485ebd79c619c3bf8341 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:48.733100 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bca0d75a0cb485ebd79c619c3bf8341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.735430 23458 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:48.736311 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 709006b08c044ed38e2df81795ed1613 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:48.736974 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7ea21ce0804c45ac69bf6430bcfb45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.737648 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709006b08c044ed38e2df81795ed1613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.738557 23457 tablet_service.cc:1511] Processing CreateTablet for tablet f55bd3b311de43ba8aca6be22937f68a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:48.739857 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55bd3b311de43ba8aca6be22937f68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.742633 23455 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:48.743952 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.746628 23454 tablet_service.cc:1511] Processing CreateTablet for tablet a59619d146f04895b0e204f6e4bdcf47 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:48.747485 23379 tablet_service.cc:1511] Processing CreateTablet for tablet d736b5901fbd4de09b83cc09a8d00fd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:48.748087 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a59619d146f04895b0e204f6e4bdcf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.750584 23380 tablet_service.cc:1511] Processing CreateTablet for tablet a59619d146f04895b0e204f6e4bdcf47 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:48.751458 23378 tablet_service.cc:1511] Processing CreateTablet for tablet 2984a5ac8b6b49c9aa8eb969a484d610 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:48.753010 23453 tablet_service.cc:1511] Processing CreateTablet for tablet d736b5901fbd4de09b83cc09a8d00fd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:48.754379 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d736b5901fbd4de09b83cc09a8d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.754586 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 2984a5ac8b6b49c9aa8eb969a484d610 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:48.755812 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2984a5ac8b6b49c9aa8eb969a484d610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.735447 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 19d9facb5378417aaa3279b3dd9f02a9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:48.762385 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d9facb5378417aaa3279b3dd9f02a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.736023 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7ea21ce0804c45ac69bf6430bcfb45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.766016 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a59619d146f04895b0e204f6e4bdcf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.766920 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.770069 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709006b08c044ed38e2df81795ed1613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.777803 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d736b5901fbd4de09b83cc09a8d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.781787 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2984a5ac8b6b49c9aa8eb969a484d610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.822710 23311 tablet_service.cc:1511] Processing CreateTablet for tablet 1bca0d75a0cb485ebd79c619c3bf8341 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:12:48.824457 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bca0d75a0cb485ebd79c619c3bf8341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.826675 23310 tablet_service.cc:1511] Processing CreateTablet for tablet 19d9facb5378417aaa3279b3dd9f02a9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:12:48.828117 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d9facb5378417aaa3279b3dd9f02a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.830672 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:12:48.832034 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7ea21ce0804c45ac69bf6430bcfb45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.834583 23308 tablet_service.cc:1511] Processing CreateTablet for tablet f55bd3b311de43ba8aca6be22937f68a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:12:48.835922 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f55bd3b311de43ba8aca6be22937f68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.838647 23307 tablet_service.cc:1511] Processing CreateTablet for tablet 709006b08c044ed38e2df81795ed1613 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:12:48.839221 23305 tablet_service.cc:1511] Processing CreateTablet for tablet a59619d146f04895b0e204f6e4bdcf47 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:12:48.840025 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709006b08c044ed38e2df81795ed1613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.845520 23306 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:12:48.846969 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.849905 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a59619d146f04895b0e204f6e4bdcf47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.855005 23304 tablet_service.cc:1511] Processing CreateTablet for tablet d736b5901fbd4de09b83cc09a8d00fd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:12:48.859083 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d736b5901fbd4de09b83cc09a8d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.860740 23378 tablet_service.cc:1511] Processing CreateTablet for tablet 6d83783f67774af1b007547d0aca8da1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:48.863070 23456 tablet_service.cc:1511] Processing CreateTablet for tablet ea456647290e4959b61a6fa23ecfbfd4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:48.863838 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:48.867491 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea456647290e4959b61a6fa23ecfbfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.864682 23380 tablet_service.cc:1511] Processing CreateTablet for tablet ea456647290e4959b61a6fa23ecfbfd4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:48.864810 23451 tablet_service.cc:1511] Processing CreateTablet for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:48.867672 23377 tablet_service.cc:1511] Processing CreateTablet for tablet 2349832eeb6f4824800ff39a557daeb6 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:48.874382 23451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.868412 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.877858 23377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2349832eeb6f4824800ff39a557daeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.878649 23302 tablet_service.cc:1511] Processing CreateTablet for tablet 6d83783f67774af1b007547d0aca8da1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:48.866371 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 6d83783f67774af1b007547d0aca8da1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:12:48.881002 23302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83783f67774af1b007547d0aca8da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.881693 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea456647290e4959b61a6fa23ecfbfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.882596 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83783f67774af1b007547d0aca8da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.886000 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 2984a5ac8b6b49c9aa8eb969a484d610 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:12:48.887357 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2984a5ac8b6b49c9aa8eb969a484d610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.902786 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 2349832eeb6f4824800ff39a557daeb6 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:48.905128 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2349832eeb6f4824800ff39a557daeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.913522 23510 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:48.901022 23381 tablet_service.cc:1511] Processing CreateTablet for tablet f6161f3a92cf44138f9a68a6f005c4e9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:48.943136 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83783f67774af1b007547d0aca8da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.954280 23308 tablet_service.cc:1511] Processing CreateTablet for tablet ea456647290e4959b61a6fa23ecfbfd4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:12:48.955601 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea456647290e4959b61a6fa23ecfbfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.958751 23386 tablet_service.cc:1511] Processing CreateTablet for tablet 487ee45f93ab48478b7fa939ccf7f658 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:48.959868 23511 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:48.966290 23382 tablet_service.cc:1511] Processing CreateTablet for tablet 746c120ff6e048eaa7b3a03be2b8a91e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:48.968011 23305 tablet_service.cc:1511] Processing CreateTablet for tablet 487ee45f93ab48478b7fa939ccf7f658 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:48.973803 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:12:48.974817 23386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 487ee45f93ab48478b7fa939ccf7f658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.974874 23453 tablet_service.cc:1511] Processing CreateTablet for tablet f6161f3a92cf44138f9a68a6f005c4e9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:48.976477 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6161f3a92cf44138f9a68a6f005c4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.976639 23457 tablet_service.cc:1511] Processing CreateTablet for tablet 487ee45f93ab48478b7fa939ccf7f658 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:12:48.968380 23384 tablet_service.cc:1511] Processing CreateTablet for tablet dbf1b9383e214ea7bb12a7f8784bffd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:48.974255 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 487ee45f93ab48478b7fa939ccf7f658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.979768 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746c120ff6e048eaa7b3a03be2b8a91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.989533 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6161f3a92cf44138f9a68a6f005c4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.980455 23512 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:48.986348 23385 tablet_service.cc:1511] Processing CreateTablet for tablet 57b150e0c7494c95beb04642f466f450 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:48.991590 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b150e0c7494c95beb04642f466f450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.992183 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88fa82a6fcc0481fa12dd9a6f1782ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.994033 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf1b9383e214ea7bb12a7f8784bffd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:48.997012 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 746c120ff6e048eaa7b3a03be2b8a91e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:49.002111 23379 tablet_service.cc:1511] Processing CreateTablet for tablet 87e270393fc841ddac0d9170b0bc6554 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:49.003516 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e270393fc841ddac0d9170b0bc6554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.009599 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 487ee45f93ab48478b7fa939ccf7f658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.012933 23383 tablet_service.cc:1511] Processing CreateTablet for tablet c4de7d7863704646b55025720cc0da9c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:49.016175 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746c120ff6e048eaa7b3a03be2b8a91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.017582 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 57b150e0c7494c95beb04642f466f450 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:49.019172 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b150e0c7494c95beb04642f466f450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.021996 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4de7d7863704646b55025720cc0da9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.030723 23454 tablet_service.cc:1511] Processing CreateTablet for tablet dbf1b9383e214ea7bb12a7f8784bffd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:49.032517 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf1b9383e214ea7bb12a7f8784bffd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.054988 23510 tablet_bootstrap.cc:654] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.080165 23458 tablet_service.cc:1511] Processing CreateTablet for tablet c4de7d7863704646b55025720cc0da9c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:49.083174 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 2349832eeb6f4824800ff39a557daeb6 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:12:49.084483 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2349832eeb6f4824800ff39a557daeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.089061 23380 tablet_service.cc:1511] Processing CreateTablet for tablet fb9a5906264b4cae9c8e888ad57c13c8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:49.090476 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb9a5906264b4cae9c8e888ad57c13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.097962 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4de7d7863704646b55025720cc0da9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.102020 23310 tablet_service.cc:1511] Processing CreateTablet for tablet dbf1b9383e214ea7bb12a7f8784bffd1 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:12:49.115058 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf1b9383e214ea7bb12a7f8784bffd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.124080 23455 tablet_service.cc:1511] Processing CreateTablet for tablet cfaa0d3f687341a1ac860baa49e8bfb4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:49.078869 23511 tablet_bootstrap.cc:654] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.065011 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 87e270393fc841ddac0d9170b0bc6554 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:49.111199 23510 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
W20260709 12:12:49.113149 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:49.126246 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e270393fc841ddac0d9170b0bc6554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.102334 23378 tablet_service.cc:1511] Processing CreateTablet for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:49.126688 23452 tablet_service.cc:1511] Processing CreateTablet for tablet ba09297a7e5c4b4f8b873c36c70e2a2d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:49.127758 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ebaab4fff4457a89f5a00d4f71ddc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.127921 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba09297a7e5c4b4f8b873c36c70e2a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.128775 23304 tablet_service.cc:1511] Processing CreateTablet for tablet 87e270393fc841ddac0d9170b0bc6554 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:12:49.126679 23510 ts_tablet_manager.cc:1403] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.233s	user 0.014s	sys 0.005s
I20260709 12:12:49.129758 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfaa0d3f687341a1ac860baa49e8bfb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.130019 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e270393fc841ddac0d9170b0bc6554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.115808 23311 tablet_service.cc:1511] Processing CreateTablet for tablet f6161f3a92cf44138f9a68a6f005c4e9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:12:49.132247 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6161f3a92cf44138f9a68a6f005c4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.132755 23383 tablet_service.cc:1511] Processing CreateTablet for tablet cfaa0d3f687341a1ac860baa49e8bfb4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:49.122274 23512 tablet_bootstrap.cc:654] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.134712 23456 tablet_service.cc:1511] Processing CreateTablet for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:49.136423 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ebaab4fff4457a89f5a00d4f71ddc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.164240 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 57b150e0c7494c95beb04642f466f450 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:12:49.120971 23451 tablet_service.cc:1511] Processing CreateTablet for tablet fb9a5906264b4cae9c8e888ad57c13c8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:49.165822 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b150e0c7494c95beb04642f466f450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.145176 23307 tablet_service.cc:1511] Processing CreateTablet for tablet c4de7d7863704646b55025720cc0da9c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:12:49.150070 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfaa0d3f687341a1ac860baa49e8bfb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.150770 23510 raft_consensus.cc:359] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.153533 23377 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe5680eae64fa2a2565e9b0973703d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:49.166448 23451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb9a5906264b4cae9c8e888ad57c13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.148669 23386 tablet_service.cc:1511] Processing CreateTablet for tablet ba09297a7e5c4b4f8b873c36c70e2a2d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:49.172080 23510 raft_consensus.cc:385] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.179754 23306 tablet_service.cc:1511] Processing CreateTablet for tablet fb9a5906264b4cae9c8e888ad57c13c8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:12:49.182830 23379 tablet_service.cc:1511] Processing CreateTablet for tablet abd134c48fc14e60af68e7bff27733c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:49.190775 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4de7d7863704646b55025720cc0da9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.187674 23510 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.176399 23302 tablet_service.cc:1511] Processing CreateTablet for tablet 746c120ff6e048eaa7b3a03be2b8a91e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:12:49.212884 23302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746c120ff6e048eaa7b3a03be2b8a91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.214377 23303 tablet_service.cc:1511] Processing CreateTablet for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:12:49.219647 23377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe5680eae64fa2a2565e9b0973703d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.225834 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb9a5906264b4cae9c8e888ad57c13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.224601 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd134c48fc14e60af68e7bff27733c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.239157 23305 tablet_service.cc:1511] Processing CreateTablet for tablet cfaa0d3f687341a1ac860baa49e8bfb4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:12:49.244120 23457 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe5680eae64fa2a2565e9b0973703d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:49.245608 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe5680eae64fa2a2565e9b0973703d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.246194 23382 tablet_service.cc:1511] Processing CreateTablet for tablet 9d236b23181b45e5a16e91a3b6a30058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:49.246831 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 29bc5916fff44ea3b1dcd31091d822b9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:49.249315 23512 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:49.199177 23453 tablet_service.cc:1511] Processing CreateTablet for tablet 9d236b23181b45e5a16e91a3b6a30058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:49.269986 23512 ts_tablet_manager.cc:1403] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.290s	user 0.026s	sys 0.000s
I20260709 12:12:49.270035 23459 tablet_service.cc:1511] Processing CreateTablet for tablet ca56a943ba904b779d7584f94c266344 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:49.249547 23451 tablet_service.cc:1511] Processing CreateTablet for tablet abd134c48fc14e60af68e7bff27733c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:49.273236 23512 raft_consensus.cc:359] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.274103 23512 raft_consensus.cc:385] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.274456 23512 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.275409 23512 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.276286 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 446014759a9c4a0f9c99759951367083 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:49.277793 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d236b23181b45e5a16e91a3b6a30058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.272476 23378 tablet_service.cc:1511] Processing CreateTablet for tablet 271ce3c0dde64506a60105c55526e4a3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:49.278906 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446014759a9c4a0f9c99759951367083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.285369 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca56a943ba904b779d7584f94c266344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.293805 23451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd134c48fc14e60af68e7bff27733c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.301941 23304 tablet_service.cc:1511] Processing CreateTablet for tablet dcbe5680eae64fa2a2565e9b0973703d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:12:49.250226 23310 tablet_service.cc:1511] Processing CreateTablet for tablet ba09297a7e5c4b4f8b873c36c70e2a2d (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:12:49.252226 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 29bc5916fff44ea3b1dcd31091d822b9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:49.303921 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bc5916fff44ea3b1dcd31091d822b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.306273 23346 heartbeater.cc:499] Master 127.21.150.254:44861 was elected leader, sending a full tablet report...
I20260709 12:12:49.254076 23386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba09297a7e5c4b4f8b873c36c70e2a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.309948 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271ce3c0dde64506a60105c55526e4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.253800 23458 tablet_service.cc:1511] Processing CreateTablet for tablet c192a749705545f298d61bbb8ccb5665 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:49.254496 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ebaab4fff4457a89f5a00d4f71ddc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.312886 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c192a749705545f298d61bbb8ccb5665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.315534 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d236b23181b45e5a16e91a3b6a30058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.255766 23381 tablet_service.cc:1511] Processing CreateTablet for tablet c192a749705545f298d61bbb8ccb5665 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:49.256012 23385 tablet_service.cc:1511] Processing CreateTablet for tablet ca56a943ba904b779d7584f94c266344 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:49.232383 23510 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.261183 23511 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:49.319263 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bc5916fff44ea3b1dcd31091d822b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.319648 23511 ts_tablet_manager.cc:1403] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.360s	user 0.018s	sys 0.000s
I20260709 12:12:49.319936 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c192a749705545f298d61bbb8ccb5665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.322494 23511 raft_consensus.cc:359] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.323226 23511 raft_consensus.cc:385] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.323518 23511 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.325245 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca56a943ba904b779d7584f94c266344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.329715 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba09297a7e5c4b4f8b873c36c70e2a2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.336099 23460 tablet_service.cc:1511] Processing CreateTablet for tablet 271ce3c0dde64506a60105c55526e4a3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:49.262435 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 446014759a9c4a0f9c99759951367083 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:49.266330 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 22fc2096e78f47158bbb4bae3a474e2c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:49.342532 23454 tablet_service.cc:1511] Processing CreateTablet for tablet de7238eb389343cd954cb756bed5925f (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:49.346050 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446014759a9c4a0f9c99759951367083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.346699 23311 tablet_service.cc:1511] Processing CreateTablet for tablet abd134c48fc14e60af68e7bff27733c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:12:49.349591 23511 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.351564 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfaa0d3f687341a1ac860baa49e8bfb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.357580 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271ce3c0dde64506a60105c55526e4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.360323 23307 tablet_service.cc:1511] Processing CreateTablet for tablet 9d236b23181b45e5a16e91a3b6a30058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:12:49.361713 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7238eb389343cd954cb756bed5925f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.363468 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 29bc5916fff44ea3b1dcd31091d822b9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:12:49.376929 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcbe5680eae64fa2a2565e9b0973703d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.387235 23518 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.393525 23459 tablet_service.cc:1511] Processing CreateTablet for tablet 1399c0b792a048cda2bcae40b23432c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:49.393721 23518 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.394912 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1399c0b792a048cda2bcae40b23432c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.396358 23379 tablet_service.cc:1511] Processing CreateTablet for tablet 1399c0b792a048cda2bcae40b23432c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:49.401552 23421 heartbeater.cc:499] Master 127.21.150.254:44861 was elected leader, sending a full tablet report...
I20260709 12:12:49.423259 23510 ts_tablet_manager.cc:1434] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.293s	user 0.010s	sys 0.008s
I20260709 12:12:49.425722 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1399c0b792a048cda2bcae40b23432c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.427974 23453 tablet_service.cc:1511] Processing CreateTablet for tablet b055a413d01c460895f3bc5034afc461 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:49.419523 23455 tablet_service.cc:1511] Processing CreateTablet for tablet 22fc2096e78f47158bbb4bae3a474e2c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:49.420194 23518 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:49.443011 23308 tablet_service.cc:1511] Processing CreateTablet for tablet c192a749705545f298d61bbb8ccb5665 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:12:49.429371 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b055a413d01c460895f3bc5034afc461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.452179 23452 tablet_service.cc:1511] Processing CreateTablet for tablet 06d137914eb14302bb888a191819fb75 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:12:49.456030 23302 tablet_service.cc:1511] Processing CreateTablet for tablet ca56a943ba904b779d7584f94c266344 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:12:49.456619 23451 tablet_service.cc:1511] Processing CreateTablet for tablet 97dc7e2c632a45499346d6fcf8e4aee9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:49.463471 23510 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:49.443076 23512 ts_tablet_manager.cc:1434] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.173s	user 0.008s	sys 0.005s
I20260709 12:12:49.478974 23512 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:49.481910 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8cb992d1f8470a8b883e9287064025 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:49.496747 23495 heartbeater.cc:499] Master 127.21.150.254:44861 was elected leader, sending a full tablet report...
I20260709 12:12:49.505246 23377 tablet_service.cc:1511] Processing CreateTablet for tablet 22fc2096e78f47158bbb4bae3a474e2c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:12:49.506080 23382 tablet_service.cc:1511] Processing CreateTablet for tablet b055a413d01c460895f3bc5034afc461 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:49.506647 23377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22fc2096e78f47158bbb4bae3a474e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.507298 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b055a413d01c460895f3bc5034afc461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.511421 23379 tablet_service.cc:1511] Processing CreateTablet for tablet 97dc7e2c632a45499346d6fcf8e4aee9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:49.512662 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dc7e2c632a45499346d6fcf8e4aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.541383 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 446014759a9c4a0f9c99759951367083 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:12:49.546476 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bc5916fff44ea3b1dcd31091d822b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.566012 23383 tablet_service.cc:1511] Processing CreateTablet for tablet de7238eb389343cd954cb756bed5925f (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:49.575201 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7238eb389343cd954cb756bed5925f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.595424 23510 tablet_bootstrap.cc:654] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:49.598706 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c192a749705545f298d61bbb8ccb5665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.599146 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22fc2096e78f47158bbb4bae3a474e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.600672 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8cb992d1f8470a8b883e9287064025 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:49.602269 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d236b23181b45e5a16e91a3b6a30058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.605708 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446014759a9c4a0f9c99759951367083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.605875 23516 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.606343 23516 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.611795 23516 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:49.636335 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8cb992d1f8470a8b883e9287064025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.639931 23511 ts_tablet_manager.cc:1434] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.320s	user 0.060s	sys 0.100s
I20260709 12:12:49.645839 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd134c48fc14e60af68e7bff27733c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.652494 23451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dc7e2c632a45499346d6fcf8e4aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.669646 23524 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.670189 23524 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:49.673724 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22fc2096e78f47158bbb4bae3a474e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.677809 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d137914eb14302bb888a191819fb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.680682 23385 tablet_service.cc:1511] Processing CreateTablet for tablet a658e02588314c9ca8a73f5eb423fb27 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:49.681762 23511 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:49.683714 23381 tablet_service.cc:1511] Processing CreateTablet for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:49.685012 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.684736 23379 tablet_service.cc:1511] Processing CreateTablet for tablet c5d8af85525845a192206b43a61f5c41 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:49.685328 23458 tablet_service.cc:1511] Processing CreateTablet for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:49.686116 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a658e02588314c9ca8a73f5eb423fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.681766 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8cb992d1f8470a8b883e9287064025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.686335 23302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca56a943ba904b779d7584f94c266344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.704044 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d8af85525845a192206b43a61f5c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.704499 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.637208 23457 tablet_service.cc:1511] Processing CreateTablet for tablet 5e2145c1046c436e8ca4a54ef0df4ab5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:49.693220 23378 tablet_service.cc:1511] Processing CreateTablet for tablet 5e2145c1046c436e8ca4a54ef0df4ab5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:49.702494 23524 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:49.693221 23454 tablet_service.cc:1511] Processing CreateTablet for tablet a658e02588314c9ca8a73f5eb423fb27 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:49.763816 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a658e02588314c9ca8a73f5eb423fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.769331 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e2145c1046c436e8ca4a54ef0df4ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.771471 23382 tablet_service.cc:1511] Processing CreateTablet for tablet c3d1af6a7ce14f5387824508ce04bfd0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:49.773028 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d1af6a7ce14f5387824508ce04bfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.777953 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e2145c1046c436e8ca4a54ef0df4ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.778075 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 71b8a974ecc34bd487fe9c208e6acc17 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:49.779744 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b8a974ecc34bd487fe9c208e6acc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.743978 23310 tablet_service.cc:1511] Processing CreateTablet for tablet de7238eb389343cd954cb756bed5925f (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:12:49.805238 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7238eb389343cd954cb756bed5925f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.814193 23510 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:49.815178 23386 tablet_service.cc:1511] Processing CreateTablet for tablet 06d137914eb14302bb888a191819fb75 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:12:49.821288 23386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d137914eb14302bb888a191819fb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.835507 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:49.837366 23362 leader_election.cc:343] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:49.828755 23510 ts_tablet_manager.cc:1403] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.365s	user 0.014s	sys 0.019s
I20260709 12:12:49.847043 23512 tablet_bootstrap.cc:654] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:49.873944 23206 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 64084us
I20260709 12:12:49.879829 23510 raft_consensus.cc:359] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:49.881909 23204 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47767.6us
I20260709 12:12:49.891458 23453 tablet_service.cc:1511] Processing CreateTablet for tablet c3d1af6a7ce14f5387824508ce04bfd0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:49.893402 23453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d1af6a7ce14f5387824508ce04bfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.925539 23510 raft_consensus.cc:385] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:49.932721 23510 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:49.927044 23306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 27) took 1024 ms. Trace:
I20260709 12:12:49.930172 23305 tablet_service.cc:1511] Processing CreateTablet for tablet 271ce3c0dde64506a60105c55526e4a3 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:12:49.952971 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 71b8a974ecc34bd487fe9c208e6acc17 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:49.954960 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b8a974ecc34bd487fe9c208e6acc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.978395 23459 tablet_service.cc:1511] Processing CreateTablet for tablet c5d8af85525845a192206b43a61f5c41 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:49.980345 23459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d8af85525845a192206b43a61f5c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.979893 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 32256a541626459187bbe0f5f7e6643c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:49.981786 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32256a541626459187bbe0f5f7e6643c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:49.935664 23306 rpcz_store.cc:276] 0709 12:12:48.902702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.362552 (+459850us) service_pool.cc:224] Handling call
0709 12:12:49.364321 (+  1769us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.364414 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.640487 (+276073us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:49.926348 (+285861us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:49.926962 (+   614us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.994629 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 1399c0b792a048cda2bcae40b23432c0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:12:50.001132 23511 tablet_bootstrap.cc:654] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.009691 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 2cd3243d8c2d447ca873f0890f2c0058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:12:50.014489 23452 tablet_service.cc:1511] Processing CreateTablet for tablet e3dffd4f7c72413b84a3862372dcce85 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:50.024626 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dffd4f7c72413b84a3862372dcce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.025816 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.026490 23321 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:50.028942 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.037791 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 271ce3c0dde64506a60105c55526e4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.038668 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 32256a541626459187bbe0f5f7e6643c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:50.039974 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32256a541626459187bbe0f5f7e6643c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.058550 23307 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 26) took 1156 ms. Trace:
I20260709 12:12:50.072679 23307 rpcz_store.cc:276] 0709 12:12:48.902398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.359333 (+456935us) service_pool.cc:224] Handling call
0709 12:12:49.361129 (+  1796us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.361220 (+    91us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.779871 (+418651us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.058005 (+278134us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.058478 (+   473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:49.935657 23510 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.070616 23457 tablet_service.cc:1511] Processing CreateTablet for tablet fa5f71c212664b78a4deb373bf5ed2f5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:50.071717 23303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 30) took 1168 ms. Trace:
I20260709 12:12:50.072381 23512 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.075378 23379 tablet_service.cc:1511] Processing CreateTablet for tablet fa5f71c212664b78a4deb373bf5ed2f5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:50.064903 23304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 24) took 1163 ms. Trace:
I20260709 12:12:50.089514 23460 tablet_service.cc:1511] Processing CreateTablet for tablet baab486079784de4933d5f96a0a5321b (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:50.079914 23303 rpcz_store.cc:276] 0709 12:12:48.903997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.540059 (+636062us) service_pool.cc:224] Handling call
0709 12:12:49.570729 (+ 30670us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.570873 (+   144us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.833944 (+263071us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.071232 (+237288us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.071651 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.095613 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:50.096312 23396 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:50.097761 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:50.098587 23516 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.098953 23516 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.099289 23516 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.099279 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 97dc7e2c632a45499346d6fcf8e4aee9 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:12:50.100816 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dc7e2c632a45499346d6fcf8e4aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.082394 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5f71c212664b78a4deb373bf5ed2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.082472 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.082504 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.083148 23311 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 25) took 1181 ms. Trace:
W20260709 12:12:50.107606 23287 leader_election.cc:343] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:50.107713 23434 leader_election.cc:343] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.107839 23311 rpcz_store.cc:276] 0709 12:12:48.902073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.345578 (+443505us) service_pool.cc:224] Handling call
0709 12:12:49.347596 (+  2018us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.347682 (+    86us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.818059 (+470377us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.082671 (+264612us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.083086 (+   415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.082870 23512 ts_tablet_manager.cc:1403] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.604s	user 0.013s	sys 0.013s
I20260709 12:12:50.080379 23455 tablet_service.cc:1511] Processing CreateTablet for tablet 19b18a93b6e8478e9f44ce88b342ab54 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:12:50.110437 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19b18a93b6e8478e9f44ce88b342ab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.110613 23454 tablet_service.cc:1511] Processing CreateTablet for tablet ec46374214e6444fa4c5afabe5bb309a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:12:50.111873 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec46374214e6444fa4c5afabe5bb309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.112323 23512 raft_consensus.cc:359] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.113082 23516 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.114811 23311 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8cb992d1f8470a8b883e9287064025 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:12:50.113058 23512 raft_consensus.cc:385] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.118146 23512 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.117821 23385 tablet_service.cc:1511] Processing CreateTablet for tablet 2cd3243d8c2d447ca873f0890f2c0058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:12:50.119359 23308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 28) took 1216 ms. Trace:
I20260709 12:12:50.119004 23512 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.118193 23309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 33) took 1208 ms. Trace:
I20260709 12:12:50.119855 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:50.123207 23535 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.123761 23396 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.123606 23451 tablet_service.cc:1511] Processing CreateTablet for tablet 9689b2f60f934eb6b1c067507423b816 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:50.124665 23377 tablet_service.cc:1511] Processing CreateTablet for tablet e3dffd4f7c72413b84a3862372dcce85 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:50.120129 23308 rpcz_store.cc:276] 0709 12:12:48.903005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.441978 (+538973us) service_pool.cc:224] Handling call
0709 12:12:49.444107 (+  2129us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.444229 (+   122us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.962804 (+518575us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.118744 (+155940us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.119300 (+   556us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:50.121104 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 92375.5us
W20260709 12:12:50.132851 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:50.138411 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8cb992d1f8470a8b883e9287064025. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.140288 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1399c0b792a048cda2bcae40b23432c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.133773 23396 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:50.146904 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 19b18a93b6e8478e9f44ce88b342ab54 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:12:50.148214 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:50.151019 23511 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:50.151547 23511 ts_tablet_manager.cc:1403] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.483s	user 0.012s	sys 0.013s
I20260709 12:12:50.154472 23511 raft_consensus.cc:359] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.155191 23511 raft_consensus.cc:385] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.155498 23511 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.160458 23510 ts_tablet_manager.cc:1434] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.320s	user 0.009s	sys 0.007s
W20260709 12:12:50.161574 23436 leader_election.cc:343] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.162070 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:50.162678 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.162873 23524 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:50.162817 23308 tablet_service.cc:1511] Processing CreateTablet for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:12:50.165251 23516 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:50.166399 23287 leader_election.cc:343] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.137660 23460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baab486079784de4933d5f96a0a5321b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.173581 23386 tablet_service.cc:1511] Processing CreateTablet for tablet baab486079784de4933d5f96a0a5321b (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:50.087538 23304 rpcz_store.cc:276] 0709 12:12:48.901728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.300956 (+399228us) service_pool.cc:224] Handling call
0709 12:12:49.302824 (+  1868us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.302973 (+   149us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.941760 (+638787us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.064267 (+122507us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.064807 (+   540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.175530 23310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 31) took 1271 ms. Trace:
I20260709 12:12:50.176234 23304 tablet_service.cc:1511] Processing CreateTablet for tablet a658e02588314c9ca8a73f5eb423fb27 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:12:50.175993 23310 rpcz_store.cc:276] 0709 12:12:48.904349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.592882 (+688533us) service_pool.cc:224] Handling call
0709 12:12:49.804743 (+211861us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.804848 (+   105us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.066854 (+262006us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.175043 (+108189us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.175476 (+   433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.179106 23310 tablet_service.cc:1511] Processing CreateTablet for tablet 06d137914eb14302bb888a191819fb75 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:12:50.181543 23382 tablet_service.cc:1511] Processing CreateTablet for tablet ec46374214e6444fa4c5afabe5bb309a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 12:12:50.182919 23422 tablet.cc:2406] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:12:50.183305 23302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 29) took 1280 ms. Trace:
I20260709 12:12:50.183688 23302 rpcz_store.cc:276] 0709 12:12:48.903334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.455040 (+551706us) service_pool.cc:224] Handling call
0709 12:12:49.456886 (+  1846us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.456980 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.912295 (+455315us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.169537 (+257242us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.183238 (+ 13701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.121836 23309 rpcz_store.cc:276] 0709 12:12:48.909966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.265394 (+355428us) service_pool.cc:224] Handling call
0709 12:12:49.345550 (+ 80156us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.345661 (+   111us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:49.757358 (+411697us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.117689 (+360331us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.118135 (+   446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.186097 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.187331 23516 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.187960 23516 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:50.188928 23516 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.189180 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 5e2145c1046c436e8ca4a54ef0df4ab5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:12:50.189961 23305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 32) took 1285 ms. Trace:
I20260709 12:12:50.094264 23307 tablet_service.cc:1511] Processing CreateTablet for tablet b055a413d01c460895f3bc5034afc461 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:12:50.190869 23383 tablet_service.cc:1511] Processing CreateTablet for tablet 9689b2f60f934eb6b1c067507423b816 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:50.171983 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5f71c212664b78a4deb373bf5ed2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.165493 23535 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.192318 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19b18a93b6e8478e9f44ce88b342ab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.190405 23305 rpcz_store.cc:276] 0709 12:12:48.904861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.832961 (+928100us) service_pool.cc:224] Handling call
0709 12:12:50.016278 (+183317us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.019905 (+  3627us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.120095 (+100190us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.163975 (+ 43880us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.189891 (+ 25916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.195147 23305 tablet_service.cc:1511] Processing CreateTablet for tablet c5d8af85525845a192206b43a61f5c41 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:12:50.198359 23385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cd3243d8c2d447ca873f0890f2c0058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.199596 23512 ts_tablet_manager.cc:1434] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.009s
I20260709 12:12:50.200783 23377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dffd4f7c72413b84a3862372dcce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.201596 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a658e02588314c9ca8a73f5eb423fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.202768 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.205824 23302 tablet_service.cc:1511] Processing CreateTablet for tablet c3d1af6a7ce14f5387824508ce04bfd0 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:12:50.206975 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b055a413d01c460895f3bc5034afc461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.192520 23535 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:50.210374 23510 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.214376 23302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d1af6a7ce14f5387824508ce04bfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.217532 23386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baab486079784de4933d5f96a0a5321b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.219048 23381 tablet_service.cc:1511] Processing CreateTablet for tablet ff06250da5d340babc2a28fd336380a7 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:50.219928 23451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9689b2f60f934eb6b1c067507423b816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.220173 23383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9689b2f60f934eb6b1c067507423b816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.220453 23512 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.186265 23511 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.225469 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cd3243d8c2d447ca873f0890f2c0058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.225615 23511 ts_tablet_manager.cc:1434] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.003s
I20260709 12:12:50.226207 23378 tablet_service.cc:1511] Processing CreateTablet for tablet 350816ae62fd4fb598a9ea6c84adf933 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:50.227560 23378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350816ae62fd4fb598a9ea6c84adf933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.229760 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e2145c1046c436e8ca4a54ef0df4ab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.230978 23452 tablet_service.cc:1511] Processing CreateTablet for tablet ff06250da5d340babc2a28fd336380a7 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:50.232317 23452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff06250da5d340babc2a28fd336380a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.201294 23384 tablet_service.cc:1511] Processing CreateTablet for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:50.233745 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5d8af85525845a192206b43a61f5c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.234512 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.237895 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d137914eb14302bb888a191819fb75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.240983 23381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff06250da5d340babc2a28fd336380a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.241760 23382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec46374214e6444fa4c5afabe5bb309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.255824 23454 tablet_service.cc:1511] Processing CreateTablet for tablet 350816ae62fd4fb598a9ea6c84adf933 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:50.256477 23512 tablet_bootstrap.cc:654] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.257292 23454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350816ae62fd4fb598a9ea6c84adf933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.257644 23511 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:50.264629 23532 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.269477 23457 tablet_service.cc:1511] Processing CreateTablet for tablet d2bbbae982b14adea2da3b5b5217b18e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:50.275279 23535 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.286684 23532 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.291419 23510 tablet_bootstrap.cc:654] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.292662 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.293354 23321 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:50.297748 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:50.281821 23311 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 37) took 1292 ms. Trace:
I20260709 12:12:50.295053 23303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 36) took 1364 ms. Trace:
I20260709 12:12:50.313711 23458 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36032 (request call id 38) took 1044 ms. Trace:
I20260709 12:12:50.314155 23458 rpcz_store.cc:276] 0709 12:12:49.269402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.684461 (+415059us) service_pool.cc:224] Handling call
0709 12:12:49.686095 (+  1634us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.686193 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.127689 (+441496us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.313081 (+185392us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.313645 (+   564us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.314667 23377 tablet_service.cc:1511] Processing CreateTablet for tablet 2abf7c5535ea498fa1d3c949bcdc25ca (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:12:50.316407 23377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abf7c5535ea498fa1d3c949bcdc25ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.304019 23311 rpcz_store.cc:276] 0709 12:12:48.932833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.113546 (+1180713us) service_pool.cc:224] Handling call
0709 12:12:50.115616 (+  2070us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.115719 (+   103us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.216177 (+100458us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.224947 (+  8770us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.225392 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.319587 23311 tablet_service.cc:1511] Processing CreateTablet for tablet 71b8a974ecc34bd487fe9c208e6acc17 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:12:50.321134 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b8a974ecc34bd487fe9c208e6acc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.345876 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.347016 23457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bbbae982b14adea2da3b5b5217b18e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:50.348748 23362 leader_election.cc:343] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.295786 23532 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.355134 23535 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.355711 23535 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.356047 23535 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.356364 23535 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.334601 23303 rpcz_store.cc:276] 0709 12:12:48.928954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.096542 (+1167588us) service_pool.cc:224] Handling call
0709 12:12:50.100357 (+  3815us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.100453 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.266101 (+165648us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.292597 (+ 26496us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.293035 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.358752 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.359026 23218 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:50.360143 23362 leader_election.cc:343] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.362349 23384 tablet_service.cc:1511] Processing CreateTablet for tablet d2bbbae982b14adea2da3b5b5217b18e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:50.366544 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 32256a541626459187bbe0f5f7e6643c (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:12:50.367802 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32256a541626459187bbe0f5f7e6643c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.370290 23455 tablet_service.cc:1511] Processing CreateTablet for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:50.371541 23455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.370823 23535 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.372867 23380 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52264 (request call id 48) took 1006 ms. Trace:
I20260709 12:12:50.375591 23302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 41) took 1315 ms. Trace:
I20260709 12:12:50.375963 23302 rpcz_store.cc:276] 0709 12:12:49.060688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.184799 (+1124111us) service_pool.cc:224] Handling call
0709 12:12:50.206670 (+ 21871us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.206775 (+   105us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.271327 (+ 64552us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.375106 (+103779us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.375530 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.379603 23308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 38) took 1442 ms. Trace:
I20260709 12:12:50.380144 23308 rpcz_store.cc:276] 0709 12:12:48.937134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.161764 (+1224630us) service_pool.cc:224] Handling call
0709 12:12:50.163644 (+  1880us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.164773 (+  1129us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.258833 (+ 94060us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.379159 (+120326us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.379548 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.370939 23307 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 35) took 1446 ms. Trace:
I20260709 12:12:50.384384 23307 rpcz_store.cc:276] 0709 12:12:48.924940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.077117 (+1152177us) service_pool.cc:224] Handling call
0709 12:12:50.191319 (+114202us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.191411 (+    92us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.259900 (+ 68489us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.370460 (+110560us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.370883 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.385349 23309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 42) took 1320 ms. Trace:
I20260709 12:12:50.381800 23305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 43) took 1281 ms. Trace:
I20260709 12:12:50.385582 23458 tablet_service.cc:1511] Processing CreateTablet for tablet 2abf7c5535ea498fa1d3c949bcdc25ca (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:12:50.385645 23308 tablet_service.cc:1511] Processing CreateTablet for tablet fa5f71c212664b78a4deb373bf5ed2f5 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:12:50.387040 23458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abf7c5535ea498fa1d3c949bcdc25ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.385859 23305 rpcz_store.cc:276] 0709 12:12:49.100834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.194254 (+1093420us) service_pool.cc:224] Handling call
0709 12:12:50.196045 (+  1791us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.196130 (+    85us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.280729 (+ 84599us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.381298 (+100569us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.381746 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.387790 23308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5f71c212664b78a4deb373bf5ed2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.373253 23380 rpcz_store.cc:276] 0709 12:12:49.366761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.145921 (+779160us) service_pool.cc:224] Handling call
0709 12:12:50.147812 (+  1891us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.149234 (+  1422us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.237101 (+ 87867us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.372395 (+135294us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.372803 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944}
I20260709 12:12:50.386269 23307 tablet_service.cc:1511] Processing CreateTablet for tablet e3dffd4f7c72413b84a3862372dcce85 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:12:50.386358 23456 tablet_service.cc:1511] Processing CreateTablet for tablet 03644bf5f16347c98fd9217f8a7fab79 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:12:50.391224 23307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3dffd4f7c72413b84a3862372dcce85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.391579 23456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03644bf5f16347c98fd9217f8a7fab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.385751 23309 rpcz_store.cc:276] 0709 12:12:49.065147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.188201 (+1123054us) service_pool.cc:224] Handling call
0709 12:12:50.190009 (+  1808us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.190104 (+    95us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.272464 (+ 82360us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.382969 (+110505us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.385283 (+  2314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.392746 23302 tablet_service.cc:1511] Processing CreateTablet for tablet 2cd3243d8c2d447ca873f0890f2c0058 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:12:50.389943 23305 tablet_service.cc:1511] Processing CreateTablet for tablet 19b18a93b6e8478e9f44ce88b342ab54 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:12:50.383409 23304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 39) took 1442 ms. Trace:
I20260709 12:12:50.398352 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.389966 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 03644bf5f16347c98fd9217f8a7fab79 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
W20260709 12:12:50.402103 23204 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55154us
I20260709 12:12:50.408463 23510 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.402130 23384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bbbae982b14adea2da3b5b5217b18e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:12:50.412714 23362 leader_election.cc:343] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.398675 23535 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.413954 23512 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.414438 23512 ts_tablet_manager.cc:1403] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.199s	user 0.021s	sys 0.027s
I20260709 12:12:50.417325 23512 raft_consensus.cc:359] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.418041 23512 raft_consensus.cc:385] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.418354 23512 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.419189 23512 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.411046 23309 tablet_service.cc:1511] Processing CreateTablet for tablet baab486079784de4933d5f96a0a5321b (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:12:50.420790 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03644bf5f16347c98fd9217f8a7fab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.421342 23512 ts_tablet_manager.cc:1434] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:50.419575 23386 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52264 (request call id 50) took 1029 ms. Trace:
I20260709 12:12:50.422655 23386 rpcz_store.cc:276] 0709 12:12:49.401661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.153274 (+751613us) service_pool.cc:224] Handling call
0709 12:12:50.174442 (+ 21168us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.174542 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.280989 (+106447us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.419088 (+138099us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.419517 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.424065 23453 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36032 (request call id 41) took 1150 ms. Trace:
I20260709 12:12:50.405835 23385 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52264 (request call id 49) took 1038 ms. Trace:
I20260709 12:12:50.437359 23453 rpcz_store.cc:276] 0709 12:12:49.273416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.868623 (+595207us) service_pool.cc:224] Handling call
0709 12:12:49.892702 (+ 24079us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.892852 (+   150us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.286957 (+394105us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.423586 (+136629us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.424004 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.438371 23385 rpcz_store.cc:276] 0709 12:12:49.367216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.101851 (+734635us) service_pool.cc:224] Handling call
0709 12:12:50.123227 (+ 21376us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.123341 (+   114us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.275107 (+151766us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.375732 (+100625us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.404713 (+ 28981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.404922 23304 rpcz_store.cc:276] 0709 12:12:48.941461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.175265 (+1233804us) service_pool.cc:224] Handling call
0709 12:12:50.177081 (+  1816us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.177186 (+   105us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.350027 (+172841us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.382924 (+ 32897us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.383352 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.440593 23306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 34) took 1527 ms. Trace:
W20260709 12:12:50.433586 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 135923us
I20260709 12:12:50.422163 23459 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36032 (request call id 43) took 1145 ms. Trace:
I20260709 12:12:50.440981 23306 rpcz_store.cc:276] 0709 12:12:48.913214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.993228 (+1080014us) service_pool.cc:224] Handling call
0709 12:12:50.106063 (+112835us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae6a8831 00007ff0ae685ac3 00007ff0ae634400 00007ff0a7ab9af4 00007ff0a7ab9a82 00007ff0a7ab99b1
0709 12:12:50.120246 (+ 14183us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.129766 (+  9520us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.263139 (+133373us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.413008 (+149869us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.440534 (+ 27526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46187392}
I20260709 12:12:50.441618 23459 rpcz_store.cc:276] 0709 12:12:49.276371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.869144 (+592773us) service_pool.cc:224] Handling call
0709 12:12:49.979666 (+110522us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:49.979798 (+   132us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.188306 (+208508us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.421704 (+233398us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.422103 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.443094 23306 tablet_service.cc:1511] Processing CreateTablet for tablet ec46374214e6444fa4c5afabe5bb309a (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:12:50.444903 23512 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.445255 23302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cd3243d8c2d447ca873f0890f2c0058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.446338 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:50.446625 23304 tablet_service.cc:1511] Processing CreateTablet for tablet 9689b2f60f934eb6b1c067507423b816 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:12:50.447842 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9689b2f60f934eb6b1c067507423b816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.448962 23310 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 40) took 1464 ms. Trace:
I20260709 12:12:50.453717 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19b18a93b6e8478e9f44ce88b342ab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.453873 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:50.454408 23321 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.454525 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.455358 23535 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:50.461717 23510 ts_tablet_manager.cc:1403] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.267s	user 0.003s	sys 0.019s
I20260709 12:12:50.461807 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec46374214e6444fa4c5afabe5bb309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.465165 23512 tablet_bootstrap.cc:654] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.465620 23511 tablet_bootstrap.cc:654] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.488689 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baab486079784de4933d5f96a0a5321b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.470955 23321 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:50.507340 23303 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 45) took 1308 ms. Trace:
I20260709 12:12:50.507781 23303 rpcz_store.cc:276] 0709 12:12:49.201701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.365661 (+1163960us) service_pool.cc:224] Handling call
0709 12:12:50.367321 (+  1660us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.367416 (+    95us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.489521 (+122105us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.506788 (+ 17267us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.507273 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.509699 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.510632 23535 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.511108 23535 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:50.511946 23535 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.449342 23310 rpcz_store.cc:276] 0709 12:12:48.984655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.178221 (+1193566us) service_pool.cc:224] Handling call
0709 12:12:50.179897 (+  1676us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.179993 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.419043 (+239050us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.448486 (+ 29443us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.448910 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.510179 23303 tablet_service.cc:1511] Processing CreateTablet for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:12:50.532265 23310 tablet_service.cc:1511] Processing CreateTablet for tablet ff06250da5d340babc2a28fd336380a7 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:12:50.524036 23304 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 52) took 1132 ms. Trace:
I20260709 12:12:50.534111 23311 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 44) took 1373 ms. Trace:
I20260709 12:12:50.534288 23304 rpcz_store.cc:276] 0709 12:12:49.391685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.445709 (+1054024us) service_pool.cc:224] Handling call
0709 12:12:50.447397 (+  1688us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.447497 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.478296 (+ 30799us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.521217 (+ 42921us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.523423 (+  2206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.534767 23311 rpcz_store.cc:276] 0709 12:12:49.160847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.318461 (+1157614us) service_pool.cc:224] Handling call
0709 12:12:50.320549 (+  2088us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.320674 (+   125us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.433419 (+112745us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.522622 (+ 89203us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.534026 (+ 11404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.536590 23304 tablet_service.cc:1511] Processing CreateTablet for tablet 350816ae62fd4fb598a9ea6c84adf933 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:12:50.537112 23217 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:50.538097 23304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350816ae62fd4fb598a9ea6c84adf933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.545076 23303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1376b20c4a2b47efbb5cd4afdb7f2ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.550073 23306 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 51) took 1189 ms. Trace:
I20260709 12:12:50.550650 23306 rpcz_store.cc:276] 0709 12:12:49.360480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.442105 (+1081625us) service_pool.cc:224] Handling call
0709 12:12:50.443947 (+  1842us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.444041 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.505970 (+ 61929us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.543900 (+ 37930us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.549982 (+  6082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.551340 23510 raft_consensus.cc:359] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.552387 23510 raft_consensus.cc:385] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.552804 23510 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.553376 23306 tablet_service.cc:1511] Processing CreateTablet for tablet d2bbbae982b14adea2da3b5b5217b18e (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:12:50.555264 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bbbae982b14adea2da3b5b5217b18e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.528751 23379 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:52264 (request call id 46) took 1194 ms. Trace:
I20260709 12:12:50.558890 23379 rpcz_store.cc:276] 0709 12:12:49.333157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.074396 (+741239us) service_pool.cc:224] Handling call
0709 12:12:50.105855 (+ 31459us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.120169 (+ 14314us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.355175 (+235006us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.501595 (+146420us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.527006 (+ 25411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.557960 23311 tablet_service.cc:1511] Processing CreateTablet for tablet 2abf7c5535ea498fa1d3c949bcdc25ca (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:12:50.561458 23311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abf7c5535ea498fa1d3c949bcdc25ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.562129 23524 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.562732 23524 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.564767 23524 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:50.566880 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:50.568560 23434 leader_election.cc:343] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.569875 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:50.571317 23436 leader_election.cc:343] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.571023 23510 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.571780 23436 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:50.572371 23511 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:50.572875 23511 ts_tablet_manager.cc:1403] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.323s	user 0.017s	sys 0.019s
I20260709 12:12:50.572876 23524 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:50.576753 23310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff06250da5d340babc2a28fd336380a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.577484 23305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 48) took 1342 ms. Trace:
I20260709 12:12:50.587764 23309 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 50) took 1302 ms. Trace:
I20260709 12:12:50.588685 23511 raft_consensus.cc:359] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.589578 23511 raft_consensus.cc:385] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.588773 23309 rpcz_store.cc:276] 0709 12:12:49.285021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.392726 (+1107705us) service_pool.cc:224] Handling call
0709 12:12:50.469583 (+ 76857us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.474849 (+  5266us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.539873 (+ 65024us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.576439 (+ 36566us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.587382 (+ 10943us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.589967 23511 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.590148 23308 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 46) took 1384 ms. Trace:
I20260709 12:12:50.579299 23305 rpcz_store.cc:276] 0709 12:12:49.234934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.387970 (+1153036us) service_pool.cc:224] Handling call
0709 12:12:50.409178 (+ 21208us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.440703 (+ 31525us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.484398 (+ 43695us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.576873 (+ 92475us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.577403 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.590890 23511 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.590536 23308 rpcz_store.cc:276] 0709 12:12:49.205641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.384079 (+1178438us) service_pool.cc:224] Handling call
0709 12:12:50.387365 (+  3286us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.387452 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.462523 (+ 75071us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.589614 (+127091us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.590080 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.593307 23307 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 47) took 1387 ms. Trace:
I20260709 12:12:50.593788 23309 tablet_service.cc:1511] Processing CreateTablet for tablet 03644bf5f16347c98fd9217f8a7fab79 (DEFAULT_TABLE table=test-workload [id=a60af847ced64e03baaf6f12357ca46d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:12:50.587608 23510 ts_tablet_manager.cc:1434] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.125s	user 0.007s	sys 0.002s
I20260709 12:12:50.595685 23510 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.599954 23302 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42786 (request call id 49) took 1338 ms. Trace:
I20260709 12:12:50.602481 23511 ts_tablet_manager.cc:1434] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:12:50.596730 23309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03644bf5f16347c98fd9217f8a7fab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:12:50.600543 23302 rpcz_store.cc:276] 0709 12:12:49.261908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.391880 (+1129972us) service_pool.cc:224] Handling call
0709 12:12:50.393540 (+  1660us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.393618 (+    78us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.494369 (+100751us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.599237 (+104868us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.599875 (+   638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.593726 23307 rpcz_store.cc:276] 0709 12:12:49.205984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.385381 (+1179397us) service_pool.cc:224] Handling call
0709 12:12:50.390802 (+  5421us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:12:50.390885 (+    83us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:12:50.514539 (+123654us) tablet_metadata.cc:665] Metadata flushed
0709 12:12:50.592803 (+ 78264us) tablet_replica.cc:206] Creating consensus instance
0709 12:12:50.593247 (+   444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:50.612051 23511 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:50.613539 23510 tablet_bootstrap.cc:654] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.617573 23512 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.618114 23512 ts_tablet_manager.cc:1403] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.173s	user 0.015s	sys 0.003s
I20260709 12:12:50.629490 23516 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.630209 23516 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.647473 23516 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.649343 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:50.650241 23396 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:50.651294 23512 raft_consensus.cc:359] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.651554 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.652015 23512 raft_consensus.cc:385] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.652360 23512 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
W20260709 12:12:50.653877 23287 leader_election.cc:343] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.654742 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.656101 23516 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.656562 23516 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.656997 23516 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.669021 23511 tablet_bootstrap.cc:654] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:50.673110 23287 consensus_peers.cc:597] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:50.672922 23516 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.673880 23512 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.677125 23512 ts_tablet_manager.cc:1434] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.059s	user 0.000s	sys 0.013s
I20260709 12:12:50.678187 23512 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.685119 23512 tablet_bootstrap.cc:654] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.687122 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:50.687747 23396 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.692296 23510 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.693020 23510 ts_tablet_manager.cc:1403] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.026s
I20260709 12:12:50.694031 23516 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.695465 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:50.696803 23287 leader_election.cc:343] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.697319 23396 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:50.697309 23510 raft_consensus.cc:359] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.698607 23510 raft_consensus.cc:385] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.698690 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.699117 23510 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.699718 23516 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.700145 23511 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:50.701213 23511 ts_tablet_manager.cc:1403] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.089s	user 0.023s	sys 0.020s
I20260709 12:12:50.701306 23510 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.702108 23512 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.702634 23512 ts_tablet_manager.cc:1403] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.005s
I20260709 12:12:50.702117 23516 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:50.703601 23516 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.705651 23512 raft_consensus.cc:359] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.706110 23510 ts_tablet_manager.cc:1434] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:12:50.706390 23512 raft_consensus.cc:385] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.706676 23512 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.707139 23510 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.707509 23512 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.709731 23512 ts_tablet_manager.cc:1434] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:12:50.710731 23512 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.714203 23510 tablet_bootstrap.cc:654] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.717140 23512 tablet_bootstrap.cc:654] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.704754 23511 raft_consensus.cc:359] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.722384 23511 raft_consensus.cc:385] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.722823 23511 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.723867 23511 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.726722 23510 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.727370 23510 ts_tablet_manager.cc:1403] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 12:12:50.730749 23512 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.730343 23510 raft_consensus.cc:359] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.731119 23510 raft_consensus.cc:385] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.731266 23512 ts_tablet_manager.cc:1403] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 12:12:50.731415 23510 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.732275 23510 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.734167 23512 raft_consensus.cc:359] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.734658 23510 ts_tablet_manager.cc:1434] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:50.734871 23512 raft_consensus.cc:385] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.735167 23512 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.735781 23512 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.736248 23510 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.737985 23512 ts_tablet_manager.cc:1434] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:12:50.738909 23512 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.740209 23511 ts_tablet_manager.cc:1434] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 12:12:50.745147 23512 tablet_bootstrap.cc:654] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.751513 23510 tablet_bootstrap.cc:654] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.753906 23396 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:50.756235 23536 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:50.763507 23511 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:50.765621 23510 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.766306 23510 ts_tablet_manager.cc:1403] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.006s
I20260709 12:12:50.768870 23535 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.769642 23535 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.770416 23510 raft_consensus.cc:359] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.771381 23510 raft_consensus.cc:385] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.771741 23510 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.773079 23535 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:12:50.774400 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.774791 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.775648 23470 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
W20260709 12:12:50.776242 23360 leader_election.cc:343] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.772848 23510 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.777359 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:50.779165 23540 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.779636 23540 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.780035 23540 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.782989 23510 ts_tablet_manager.cc:1434] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:12:50.784521 23510 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.784920 23524 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.785389 23524 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.791050 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.791926 23321 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:50.793147 23510 tablet_bootstrap.cc:654] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.793648 23434 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:50.795722 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:50.797752 23436 leader_election.cc:343] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.789227 23524 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:50.799077 23524 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.799535 23524 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.799945 23524 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.808331 23510 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.809048 23510 ts_tablet_manager.cc:1403] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.008s
I20260709 12:12:50.812945 23510 raft_consensus.cc:359] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.813984 23510 raft_consensus.cc:385] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.814296 23510 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.815493 23510 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.818919 23510 ts_tablet_manager.cc:1434] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:12:50.822659 23510 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:50.831401 23510 tablet_bootstrap.cc:654] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.844893 23510 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.845626 23510 ts_tablet_manager.cc:1403] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.009s
W20260709 12:12:50.852764 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59233.9us
I20260709 12:12:50.854945 23535 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.855788 23535 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.855033 23556 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.858248 23556 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.859817 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.862818 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.872997 23321 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:50.863596 23535 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.848935 23540 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.864282 23510 raft_consensus.cc:359] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.874639 23510 raft_consensus.cc:385] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.874486 23360 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:50.874897 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:50.875154 23510 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.875603 23540 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:12:50.865963 23217 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:50.876521 23540 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.876227 23510 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.876992 23540 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.877530 23540 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.877836 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.878764 23469 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.881477 23512 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:50.881389 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:50.882069 23512 ts_tablet_manager.cc:1403] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.143s	user 0.052s	sys 0.070s
I20260709 12:12:50.883777 23556 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.872007 23511 tablet_bootstrap.cc:654] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.875627 23396 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:50.885545 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.885458 23512 raft_consensus.cc:359] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.885313 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:50.885751 23540 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.886368 23468 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:50.879107 23510 ts_tablet_manager.cc:1434] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:12:50.886883 23536 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.887255 23536 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.887600 23510 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
W20260709 12:12:50.887916 23360 leader_election.cc:343] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.870599 23559 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:50.887579 23536 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.886312 23512 raft_consensus.cc:385] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.890671 23512 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.891729 23540 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.865675 23524 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.893764 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.894860 23510 tablet_bootstrap.cc:654] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.895237 23470 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:50.895843 23524 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:50.895758 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:50.888517 23559 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:50.897356 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:50.891764 23512 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.898362 23396 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:50.898193 23559 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.899554 23559 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:50.900127 23512 ts_tablet_manager.cc:1434] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:12:50.901198 23512 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:50.904012 23321 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.905649 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:50.907943 23512 tablet_bootstrap.cc:654] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:50.909523 23434 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:50.911167 23524 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:50.915633 23469 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:50.917249 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:50.918392 23540 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.918982 23540 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:50.920010 23540 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.925417 23535 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.926136 23535 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.928748 23535 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.931129 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:50.931985 23320 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:50.934128 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:50.935166 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.934680 23219 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:50.935725 23321 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
W20260709 12:12:50.936421 23362 leader_election.cc:343] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:50.937325 23540 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:50.937197 23360 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:50.937790 23540 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.938175 23540 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.938726 23535 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.943903 23536 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.948779 23536 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.958237 23511 tablet_bootstrap.cc:492] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:50.958854 23511 ts_tablet_manager.cc:1403] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.196s	user 0.019s	sys 0.026s
I20260709 12:12:50.962016 23511 raft_consensus.cc:359] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.962914 23511 raft_consensus.cc:385] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.962826 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:50.964075 23511 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.964172 23396 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.965081 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:50.965705 23469 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:50.966045 23536 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.966616 23536 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:12:50.965111 23511 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.980666 23566 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:50.982402 23535 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:50.983596 23535 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.987164 23469 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:50.988734 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:50.980592 23511 ts_tablet_manager.cc:1434] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.002s
I20260709 12:12:50.989926 23556 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:50.990401 23556 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:50.991209 23556 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.992799 23510 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:50.992944 23536 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:50.993261 23510 ts_tablet_manager.cc:1403] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.106s	user 0.012s	sys 0.038s
I20260709 12:12:50.994949 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:50.995873 23470 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:50.996003 23510 raft_consensus.cc:359] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:50.996767 23510 raft_consensus.cc:385] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:50.997032 23510 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:50.998202 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:50.999733 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.000901 23559 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.002465 23511 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.002076 23510 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.004954 23512 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.005566 23512 ts_tablet_manager.cc:1403] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.105s	user 0.020s	sys 0.031s
I20260709 12:12:51.006711 23540 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.009495 23566 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.016337 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.016263 23566 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:51.017755 23362 leader_election.cc:343] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.018573 23566 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.019714 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.020272 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:51.021087 23320 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.021802 23559 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.010943 23511 tablet_bootstrap.cc:654] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.023272 23469 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.011796 23510 ts_tablet_manager.cc:1434] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:12:51.022670 23219 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:51.021759 23362 leader_election.cc:343] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.025678 23510 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.008778 23512 raft_consensus.cc:359] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.008781 23396 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:51.032341 23512 raft_consensus.cc:385] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.032704 23512 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.033381 23512 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.034500 23540 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.035367 23512 ts_tablet_manager.cc:1434] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 12:12:51.036293 23512 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.039145 23540 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:51.050731 23510 tablet_bootstrap.cc:654] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.054849 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:51.056324 23360 leader_election.cc:343] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.056901 23219 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.057816 23511 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.058296 23511 ts_tablet_manager.cc:1403] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.016s
I20260709 12:12:51.058326 23320 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.061250 23511 raft_consensus.cc:359] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.061810 23512 tablet_bootstrap.cc:654] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.062140 23511 raft_consensus.cc:385] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.062444 23511 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.056828 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.063408 23511 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.069406 23565 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.070561 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.077929 23565 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.078421 23565 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:51.079221 23565 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.086939 23511 ts_tablet_manager.cc:1434] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:12:51.088138 23511 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.090998 23512 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.091468 23512 ts_tablet_manager.cc:1403] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.016s
I20260709 12:12:51.091596 23510 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.092020 23510 ts_tablet_manager.cc:1403] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.067s	user 0.007s	sys 0.021s
I20260709 12:12:51.094033 23512 raft_consensus.cc:359] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.094295 23510 raft_consensus.cc:359] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.094714 23512 raft_consensus.cc:385] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.094882 23510 raft_consensus.cc:385] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.095000 23512 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.095148 23510 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.095769 23512 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.095841 23510 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.096941 23511 tablet_bootstrap.cc:654] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.098282 23510 ts_tablet_manager.cc:1434] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:12:51.099265 23510 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.102304 23512 ts_tablet_manager.cc:1434] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:12:51.103250 23512 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.104497 23510 tablet_bootstrap.cc:654] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.108717 23512 tablet_bootstrap.cc:654] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.113234 23559 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.114774 23510 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.115504 23510 ts_tablet_manager.cc:1403] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 12:12:51.118240 23510 raft_consensus.cc:359] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.118919 23510 raft_consensus.cc:385] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.119205 23510 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.119884 23510 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.122067 23510 ts_tablet_manager.cc:1434] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:12:51.123306 23510 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.126653 23512 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.125977 23219 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.127125 23512 ts_tablet_manager.cc:1403] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.004s
I20260709 12:12:51.129117 23510 tablet_bootstrap.cc:654] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.129794 23512 raft_consensus.cc:359] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.130482 23512 raft_consensus.cc:385] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.131212 23512 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.132001 23512 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.144627 23510 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.145035 23516 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.145319 23510 ts_tablet_manager.cc:1403] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.003s
I20260709 12:12:51.145690 23516 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.148257 23516 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.148685 23510 raft_consensus.cc:359] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.149415 23510 raft_consensus.cc:385] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.149721 23510 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.150857 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.150857 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.152359 23396 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:51.154342 23287 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:12:51.155226 23287 leader_election.cc:343] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:51.157141 23287 consensus_peers.cc:597] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:51.157706 23510 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.160039 23510 ts_tablet_manager.cc:1434] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:12:51.157303 23516 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.160606 23516 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.160976 23516 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.163921 23510 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
W20260709 12:12:51.170186 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:51.170675 23510 tablet_bootstrap.cc:654] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.172513 23516 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.175643 23512 ts_tablet_manager.cc:1434] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 12:12:51.179934 23511 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.180277 23512 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.180446 23511 ts_tablet_manager.cc:1403] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.093s	user 0.026s	sys 0.043s
I20260709 12:12:51.183491 23511 raft_consensus.cc:359] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.184249 23511 raft_consensus.cc:385] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.184556 23511 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.185747 23511 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.186506 23512 tablet_bootstrap.cc:654] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.187955 23511 ts_tablet_manager.cc:1434] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.005s
I20260709 12:12:51.189039 23511 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.189872 23516 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.190562 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:51.190907 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.191144 23396 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:51.192318 23287 leader_election.cc:343] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.195225 23510 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.195683 23510 ts_tablet_manager.cc:1403] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.004s
I20260709 12:12:51.198293 23396 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:51.198387 23510 raft_consensus.cc:359] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.199045 23510 raft_consensus.cc:385] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.199314 23510 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.199910 23510 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.201850 23287 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.202203 23510 ts_tablet_manager.cc:1434] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:12:51.203001 23536 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.203128 23510 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.203166 23511 tablet_bootstrap.cc:654] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.203492 23536 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:51.204380 23536 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.210062 23510 tablet_bootstrap.cc:654] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.211872 23565 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.212353 23565 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.214495 23565 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.214947 23512 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.215392 23512 ts_tablet_manager.cc:1403] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 12:12:51.216288 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:51.217790 23360 leader_election.cc:343] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.218096 23512 raft_consensus.cc:359] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.218395 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.218765 23512 raft_consensus.cc:385] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.219034 23512 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.219720 23512 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:51.219863 23362 leader_election.cc:343] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.220480 23362 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.221570 23565 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.227216 23510 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.227738 23510 ts_tablet_manager.cc:1403] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.005s
I20260709 12:12:51.228304 23512 ts_tablet_manager.cc:1434] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:12:51.229210 23512 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.230392 23510 raft_consensus.cc:359] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.231032 23510 raft_consensus.cc:385] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.231277 23510 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.232116 23510 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.234426 23510 ts_tablet_manager.cc:1434] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:51.235641 23510 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.236109 23217 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.237571 23511 tablet_bootstrap.cc:492] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.238683 23511 ts_tablet_manager.cc:1403] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.050s	user 0.028s	sys 0.010s
I20260709 12:12:51.241600 23510 tablet_bootstrap.cc:654] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.238245 23512 tablet_bootstrap.cc:654] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.241345 23511 raft_consensus.cc:359] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.244735 23511 raft_consensus.cc:385] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.245091 23511 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.245898 23511 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.253326 23511 ts_tablet_manager.cc:1434] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:12:51.253712 23510 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.254240 23510 ts_tablet_manager.cc:1403] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 12:12:51.254568 23511 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.256924 23510 raft_consensus.cc:359] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.257661 23510 raft_consensus.cc:385] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.257934 23510 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.258860 23510 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.261201 23510 ts_tablet_manager.cc:1434] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:51.262084 23510 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.262236 23511 tablet_bootstrap.cc:654] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.268577 23510 tablet_bootstrap.cc:654] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.274670 23511 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.275080 23512 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.275205 23511 ts_tablet_manager.cc:1403] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:12:51.275516 23512 ts_tablet_manager.cc:1403] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.010s
I20260709 12:12:51.276017 23510 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.276446 23510 ts_tablet_manager.cc:1403] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.015s	user 0.002s	sys 0.008s
I20260709 12:12:51.277824 23512 raft_consensus.cc:359] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.278100 23511 raft_consensus.cc:359] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.278545 23512 raft_consensus.cc:385] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.278753 23511 raft_consensus.cc:385] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.278857 23512 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.279100 23511 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.278868 23510 raft_consensus.cc:359] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.279603 23510 raft_consensus.cc:385] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.279801 23512 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.279883 23510 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.279812 23511 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.281294 23510 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.282181 23512 ts_tablet_manager.cc:1434] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:12:51.283293 23510 ts_tablet_manager.cc:1434] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:51.284113 23511 ts_tablet_manager.cc:1434] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:12:51.284278 23512 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.285130 23511 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.289539 23510 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.291189 23512 tablet_bootstrap.cc:654] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.295904 23536 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.296461 23536 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.298894 23536 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.299932 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.301265 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:51.301384 23287 leader_election.cc:343] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.302028 23511 tablet_bootstrap.cc:654] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.301934 23396 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:51.303398 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.304243 23536 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.304842 23536 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.305298 23536 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.301481 23510 tablet_bootstrap.cc:654] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.314138 23536 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.317245 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:51.317847 23396 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.324641 23511 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.325110 23511 ts_tablet_manager.cc:1403] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.007s
I20260709 12:12:51.326259 23512 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.326689 23512 ts_tablet_manager.cc:1403] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.012s
I20260709 12:12:51.328438 23396 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:51.328281 23511 raft_consensus.cc:359] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.328977 23511 raft_consensus.cc:385] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.329262 23511 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.329663 23536 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.329281 23512 raft_consensus.cc:359] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.329737 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:51.330190 23512 raft_consensus.cc:385] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.330482 23512 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.330065 23511 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.331099 23536 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.331601 23536 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:51.331790 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.332309 23536 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.332720 23510 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.333195 23510 ts_tablet_manager.cc:1403] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.000s
W20260709 12:12:51.334550 23287 leader_election.cc:343] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.335793 23510 raft_consensus.cc:359] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.336445 23510 raft_consensus.cc:385] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.336690 23510 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.336738 23512 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.337386 23510 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.339152 23512 ts_tablet_manager.cc:1434] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:12:51.339483 23510 ts_tablet_manager.cc:1434] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:51.340016 23512 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.340648 23511 ts_tablet_manager.cc:1434] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:12:51.341531 23511 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.346535 23512 tablet_bootstrap.cc:654] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.347221 23511 tablet_bootstrap.cc:654] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.348853 23565 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.349330 23565 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.350319 23510 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.351498 23565 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.356648 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.357343 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.358206 23512 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.356679 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.358320 23510 tablet_bootstrap.cc:654] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.358819 23512 ts_tablet_manager.cc:1403] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:12:51.358853 23470 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.360037 23565 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.360356 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:51.361304 23566 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.361732 23566 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.362051 23566 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.372395 23566 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.362018 23512 raft_consensus.cc:359] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.374253 23512 raft_consensus.cc:385] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.374625 23512 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.375581 23512 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.378136 23524 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.378729 23512 ts_tablet_manager.cc:1434] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:12:51.378628 23524 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.379032 23511 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.379518 23511 ts_tablet_manager.cc:1403] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.006s
I20260709 12:12:51.379632 23512 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.380798 23524 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.382985 23511 raft_consensus.cc:359] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.383649 23511 raft_consensus.cc:385] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.383919 23511 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.383793 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.384508 23566 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.385062 23511 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.386777 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.387672 23511 ts_tablet_manager.cc:1434] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:12:51.385326 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:51.388619 23511 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.388751 23320 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.390743 23512 tablet_bootstrap.cc:654] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.391345 23510 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.391820 23510 ts_tablet_manager.cc:1403] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.010s
I20260709 12:12:51.394721 23510 raft_consensus.cc:359] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.395427 23510 raft_consensus.cc:385] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.395722 23510 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.396203 23217 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.396384 23510 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.399024 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:51.383846 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.401155 23469 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.402549 23510 ts_tablet_manager.cc:1434] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 12:12:51.403532 23510 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.404096 23524 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.407634 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.412173 23511 tablet_bootstrap.cc:654] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.412523 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:51.413344 23565 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.413506 23512 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.414017 23512 ts_tablet_manager.cc:1403] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 12:12:51.413857 23565 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.409459 23510 tablet_bootstrap.cc:654] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.416049 23565 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.416688 23565 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.417881 23512 raft_consensus.cc:359] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.418664 23512 raft_consensus.cc:385] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.417240 23565 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:51.419008 23512 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.419937 23512 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.423704 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.424403 23320 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.426249 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:51.428141 23362 leader_election.cc:343] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.428745 23512 ts_tablet_manager.cc:1434] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:12:51.427457 23566 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.429893 23566 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.431274 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.431908 23566 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.432121 23469 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.432866 23566 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.435890 23566 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.433395 23512 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.434206 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.437212 23321 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.434775 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.438463 23468 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.439867 23573 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.440073 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:51.420411 23565 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.436405 23566 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.440351 23573 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.443192 23573 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.443697 23573 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.444020 23573 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.444324 23573 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.444545 23535 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:51.447922 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.450271 23511 tablet_bootstrap.cc:492] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.450719 23511 ts_tablet_manager.cc:1403] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.000s
W20260709 12:12:51.451298 23362 leader_election.cc:343] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.451951 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.495849 23524 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.453258 23511 raft_consensus.cc:359] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.519284 23511 raft_consensus.cc:385] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.519130 23524 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.485293 23535 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.520249 23535 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.522193 23524 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.523453 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:51.524861 23362 leader_election.cc:343] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.519591 23511 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.526053 23512 tablet_bootstrap.cc:654] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.516157 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:51.454795 23360 leader_election.cc:343] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.525830 23511 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.527745 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.528280 23510 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.528618 23565 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.528746 23510 ts_tablet_manager.cc:1403] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.125s	user 0.010s	sys 0.014s
I20260709 12:12:51.529219 23511 ts_tablet_manager.cc:1434] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.078s	user 0.025s	sys 0.000s
I20260709 12:12:51.530165 23511 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.531411 23573 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.532657 23535 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.533617 23573 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.533860 23536 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.533957 23566 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.535454 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.535848 23566 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.535950 23469 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:51.528744 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74018.9us
I20260709 12:12:51.475993 23603 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.537772 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.537866 23603 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.538380 23566 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.538479 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.539304 23396 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:12:51.539216 23362 leader_election.cc:343] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.540305 23603 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.540805 23468 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.544678 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.550595 23510 raft_consensus.cc:359] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.551352 23510 raft_consensus.cc:385] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.551645 23510 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.544530 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.552807 23510 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.555022 23510 ts_tablet_manager.cc:1434] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 12:12:51.557828 23510 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.556600 23566 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:12:51.559628 23434 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:51.560740 23524 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.566229 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.567011 23321 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.568802 23536 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.569962 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:51.570668 23319 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.551120 23469 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.578583 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.578938 23360 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.579916 23320 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:51.580168 23566 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.580667 23566 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.580997 23566 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.582782 23434 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:51.576036 23536 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.584946 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:51.587580 23536 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.588236 23573 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.588737 23573 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:51.589896 23565 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.589702 23573 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.591639 23524 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.592283 23524 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.592626 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.593398 23396 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:51.595517 23512 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.596107 23512 ts_tablet_manager.cc:1403] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.163s	user 0.024s	sys 0.025s
I20260709 12:12:51.598569 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.599638 23319 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.600293 23512 raft_consensus.cc:359] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.600911 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.602133 23565 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.602624 23565 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:51.601256 23512 raft_consensus.cc:385] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.603749 23512 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.603451 23565 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.604776 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:51.604666 23512 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.605654 23321 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.607934 23512 ts_tablet_manager.cc:1434] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:12:51.609210 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:51.610424 23536 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.610966 23536 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.611421 23536 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.618759 23510 tablet_bootstrap.cc:654] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:51.620153 23287 leader_election.cc:343] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.621201 23536 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.623904 23511 tablet_bootstrap.cc:654] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.624127 23536 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.609684 23512 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.624735 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:51.625423 23396 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.627601 23535 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.628206 23535 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.630810 23535 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.631433 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:51.632786 23536 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.633612 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.632385 23512 tablet_bootstrap.cc:654] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.632560 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:51.645080 23362 leader_election.cc:343] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:51.646929 23287 leader_election.cc:343] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.642904 23536 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.642436 23535 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.651005 23603 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.654604 23360 leader_election.cc:343] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.665737 23360 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.667507 23540 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.667697 23466 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.654850 23535 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.671743 23535 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.673460 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.652436 23536 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:51.687327 23362 leader_election.cc:343] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.689682 23321 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.690387 23565 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:51.693770 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:51.695451 23360 leader_election.cc:343] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.695899 23360 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.696683 23565 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.679868 23603 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.702615 23510 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.703205 23510 ts_tablet_manager.cc:1403] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.146s	user 0.008s	sys 0.002s
I20260709 12:12:51.720484 23524 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.741462 23573 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.703691 23603 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.706120 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.755668 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.757261 23394 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:51.759202 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:51.759954 23396 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:51.762550 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.761857 23603 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.760808 23524 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.768015 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.768353 23510 raft_consensus.cc:359] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.778592 23510 raft_consensus.cc:385] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.779485 23510 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.770357 23603 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.755687 23566 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.770537 23516 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.783140 23510 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.782722 23603 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.774960 23536 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.760780 23573 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.795598 23536 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.796049 23536 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:51.796852 23536 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.802152 23566 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.800571 23573 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.803624 23524 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.787119 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.805709 23394 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:51.807601 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.808174 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:51.809149 23362 leader_election.cc:343] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.766481 23320 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
W20260709 12:12:51.813653 23362 leader_election.cc:343] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.788441 23516 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.818864 23516 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.821046 23510 ts_tablet_manager.cc:1434] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.118s	user 0.000s	sys 0.015s
I20260709 12:12:51.822353 23510 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:51.847931 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:51.840746 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 81613.1us
I20260709 12:12:51.851590 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.852375 23395 raft_consensus.cc:2368] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2.
I20260709 12:12:51.852443 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.853971 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:51.854516 23321 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.854187 23436 leader_election.cc:400] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2.
I20260709 12:12:51.859211 23436 leader_election.cc:403] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:51.860008 23512 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:51.860479 23524 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 2
I20260709 12:12:51.861608 23321 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:51.860481 23512 ts_tablet_manager.cc:1403] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.251s	user 0.011s	sys 0.003s
I20260709 12:12:51.853077 23217 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.871160 23566 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:51.873379 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.876791 23512 raft_consensus.cc:359] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.877580 23510 tablet_bootstrap.cc:654] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.878320 23512 raft_consensus.cc:385] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.878818 23512 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.879735 23512 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.883306 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:51.884394 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.884655 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:51.885118 23320 raft_consensus.cc:2368] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2.
I20260709 12:12:51.885941 23516 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:51.888924 23512 ts_tablet_manager.cc:1434] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:12:51.889986 23512 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:51.891626 23511 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:51.892405 23511 ts_tablet_manager.cc:1403] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.362s	user 0.032s	sys 0.038s
I20260709 12:12:51.890060 23620 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.871230 23603 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.881949 23395 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:51.899396 23620 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:51.900671 23360 leader_election.cc:343] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.903235 23466 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.903307 23465 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:51.904279 23620 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.903882 23536 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:12:51.904435 23524 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2.
I20260709 12:12:51.919813 23434 leader_election.cc:400] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2.
I20260709 12:12:51.922663 23603 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.905650 23360 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.924502 23535 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:51.925083 23360 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.925798 23535 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.926184 23566 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:51.926668 23566 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:51.926324 23535 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.928618 23511 raft_consensus.cc:359] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.929504 23511 raft_consensus.cc:385] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:51.929802 23511 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:51.930469 23511 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.927516 23566 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.932157 23511 ts_tablet_manager.cc:1434] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.039s	user 0.002s	sys 0.003s
I20260709 12:12:51.932832 23511 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:51.927543 23573 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:12:51.939431 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:51.940866 23395 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:51.938416 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46127.2us
I20260709 12:12:51.945902 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:51.947250 23362 leader_election.cc:343] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.939708 23535 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:51.963402 23603 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.963975 23603 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:51.967018 23512 tablet_bootstrap.cc:654] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.975937 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:51.976907 23318 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:51.968505 23603 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:51.981127 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:51.983083 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:51.984890 23540 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:51.985344 23540 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.985735 23540 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:51.986060 23511 tablet_bootstrap.cc:654] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:51.987723 23395 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:51.988888 23394 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:51.991395 23216 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:51.994495 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:51.995482 23603 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:51.995923 23603 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.996338 23603 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:51.990669 23287 leader_election.cc:343] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:51.999333 23510 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:51.999789 23510 ts_tablet_manager.cc:1403] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.178s	user 0.025s	sys 0.014s
I20260709 12:12:52.002830 23620 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.005116 23510 raft_consensus.cc:359] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.009174 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.009207 23627 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:52.014530 23510 raft_consensus.cc:385] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.015031 23510 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.015813 23510 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.022574 23510 ts_tablet_manager.cc:1434] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 12:12:52.023746 23510 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.024605 23524 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.025360 23524 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:52.026036 23540 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.026543 23524 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.040551 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.030927 23540 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:52.043064 23362 leader_election.cc:343] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.057741 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.057917 23573 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.069535 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.071332 23318 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.071246 23573 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.069348 23628 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:12:52.070533 23320 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:52.091075 23318 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:52.096979 23620 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.100885 23573 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.103056 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.067925 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:12:52.104971 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46830.9us
I20260709 12:12:52.105896 23628 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.106261 23628 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.106942 23628 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.110211 23511 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.112501 23512 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
W20260709 12:12:52.111479 23362 leader_election.cc:343] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.115097 23512 ts_tablet_manager.cc:1403] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.225s	user 0.036s	sys 0.049s
I20260709 12:12:52.115129 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.110814 23511 ts_tablet_manager.cc:1403] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.178s	user 0.018s	sys 0.017s
I20260709 12:12:52.113644 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.117405 23396 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:52.120584 23218 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.122200 23603 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.125178 23540 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.127810 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.123793 23628 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.129386 23540 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.129897 23540 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:52.106597 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.130972 23628 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.131961 23512 raft_consensus.cc:359] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.132624 23512 raft_consensus.cc:385] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.132918 23512 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.130754 23540 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.138482 23512 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.139838 23511 raft_consensus.cc:359] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.140486 23511 raft_consensus.cc:385] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.140739 23511 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.141875 23511 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.143453 23573 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.143757 23511 ts_tablet_manager.cc:1434] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.001s
I20260709 12:12:52.141669 23603 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.143918 23573 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.146137 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.146462 23510 tablet_bootstrap.cc:654] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.146708 23573 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.146893 23396 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:52.152004 23362 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:52.153607 23466 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:52.158195 23512 ts_tablet_manager.cc:1434] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 12:12:52.159377 23512 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.160159 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.161129 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.172257 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.172319 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.174762 23511 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.174871 23319 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:52.173157 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.177285 23467 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:12:52.177536 23321 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:52.182235 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.182992 23573 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:12:52.183379 23573 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.183735 23573 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.185987 23396 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:52.189883 23573 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.193166 23573 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.194403 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.180588 23540 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:12:52.198231 23467 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.198907 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.198437 23217 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:52.199968 23620 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.207603 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.209156 23467 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:12:52.210688 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.212383 23573 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:12:52.212843 23573 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:52.213737 23573 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.214255 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.214640 23620 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.214807 23320 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:12:52.215335 23628 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:52.215664 23628 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.216495 23628 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.218477 23511 tablet_bootstrap.cc:654] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.212996 23512 tablet_bootstrap.cc:654] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.221791 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:12:52.222158 23287 leader_election.cc:343] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.223213 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.223417 23572 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.223943 23572 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.223999 23627 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.224399 23572 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.224568 23627 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.225456 23627 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.226984 23510 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.227435 23510 ts_tablet_manager.cc:1403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.204s	user 0.015s	sys 0.001s
I20260709 12:12:52.230039 23510 raft_consensus.cc:359] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.230674 23510 raft_consensus.cc:385] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.230923 23510 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.232501 23572 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.236891 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:52.237509 23536 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.238096 23524 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.238440 23628 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:52.242556 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:52.249925 23572 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.250509 23573 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.252493 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.253573 23318 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.253690 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.253628 23216 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:52.255600 23362 leader_election.cc:343] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.257817 23524 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.258446 23524 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.261339 23318 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:52.262163 23524 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.263792 23320 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:52.263751 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.265163 23396 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:52.263748 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.266083 23510 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.267050 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.267915 23524 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.268291 23524 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.268594 23524 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.268954 23510 ts_tablet_manager.cc:1434] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.000s
I20260709 12:12:52.270021 23510 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.273653 23573 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.274205 23573 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:52.275097 23573 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.277984 23510 tablet_bootstrap.cc:654] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.283715 23466 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.283856 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.285684 23512 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.286494 23512 ts_tablet_manager.cc:1403] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.127s	user 0.010s	sys 0.004s
W20260709 12:12:52.294517 23434 leader_election.cc:343] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.292165 23524 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.309679 23217 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.312517 23524 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.314603 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:12:52.316125 23434 leader_election.cc:343] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.318671 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.319243 23396 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.329634 23396 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:52.330093 23511 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.330587 23511 ts_tablet_manager.cc:1403] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.156s	user 0.018s	sys 0.021s
I20260709 12:12:52.331027 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:52.331873 23524 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.332403 23524 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:52.333245 23524 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.333241 23511 raft_consensus.cc:359] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.334514 23511 raft_consensus.cc:385] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.334800 23511 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.335451 23511 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.337863 23511 ts_tablet_manager.cc:1434] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:12:52.338744 23511 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.342125 23510 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.342624 23510 ts_tablet_manager.cc:1403] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.073s	user 0.009s	sys 0.009s
I20260709 12:12:52.341744 23218 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.350489 23603 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.350773 23512 raft_consensus.cc:359] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.352159 23512 raft_consensus.cc:385] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.352409 23512 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.352909 23512 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.351468 23603 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.355748 23512 ts_tablet_manager.cc:1434] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.000s
I20260709 12:12:52.356104 23627 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.357939 23512 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.358836 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.361151 23603 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.361780 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.363466 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:52.364672 23511 tablet_bootstrap.cc:654] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.365715 23510 raft_consensus.cc:359] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.366477 23510 raft_consensus.cc:385] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.366796 23510 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.367662 23510 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.370026 23510 ts_tablet_manager.cc:1434] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.001s
I20260709 12:12:52.370947 23510 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.374271 23217 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.377727 23524 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.378206 23524 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.380621 23603 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:52.380889 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.381706 23321 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:52.385509 23512 tablet_bootstrap.cc:654] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.387259 23524 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.387295 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:52.388818 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.391278 23436 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:52.398108 23524 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.419140 23573 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.419641 23573 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.420887 23524 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.421917 23573 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.422206 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.422703 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:52.423626 23360 leader_election.cc:343] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:52.424170 23362 leader_election.cc:343] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.424621 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.425465 23573 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.431303 23627 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.431809 23627 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.433781 23627 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.434419 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.434643 23511 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.435009 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:52.435173 23511 ts_tablet_manager.cc:1403] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.097s	user 0.014s	sys 0.019s
I20260709 12:12:52.435756 23510 tablet_bootstrap.cc:654] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.436726 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.437008 23287 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:52.438148 23627 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.437992 23511 raft_consensus.cc:359] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.438604 23627 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.438751 23511 raft_consensus.cc:385] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.439023 23627 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.439163 23511 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.440094 23511 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:52.438056 23287 leader_election.cc:343] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.442998 23511 ts_tablet_manager.cc:1434] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:52.444906 23511 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.445830 23510 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.445688 23627 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.446264 23510 ts_tablet_manager.cc:1403] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.075s	user 0.007s	sys 0.006s
I20260709 12:12:52.450520 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.451046 23394 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.451545 23510 raft_consensus.cc:359] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.452203 23510 raft_consensus.cc:385] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.452453 23510 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.451964 23512 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.453313 23512 ts_tablet_manager.cc:1403] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.096s	user 0.012s	sys 0.003s
I20260709 12:12:52.455873 23512 raft_consensus.cc:359] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.456489 23512 raft_consensus.cc:385] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.458472 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:52.458611 23524 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.459300 23510 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.459903 23512 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.460299 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.460557 23512 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:52.462046 23287 leader_election.cc:343] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.462633 23512 ts_tablet_manager.cc:1434] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:12:52.463537 23512 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.469283 23512 tablet_bootstrap.cc:654] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.472123 23466 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.475243 23540 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.477550 23510 ts_tablet_manager.cc:1434] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.001s
I20260709 12:12:52.478860 23510 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.459656 23511 tablet_bootstrap.cc:654] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.459259 23627 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.485682 23287 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.487641 23536 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.488161 23536 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.486475 23524 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:12:52.497231 23573 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.488966 23536 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.546967 23510 tablet_bootstrap.cc:654] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.550465 23511 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.550940 23511 ts_tablet_manager.cc:1403] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.106s	user 0.013s	sys 0.005s
I20260709 12:12:52.552913 23512 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.553359 23512 ts_tablet_manager.cc:1403] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.090s	user 0.021s	sys 0.029s
I20260709 12:12:52.554602 23511 raft_consensus.cc:359] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.555315 23511 raft_consensus.cc:385] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.555615 23511 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.555730 23512 raft_consensus.cc:359] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.556427 23512 raft_consensus.cc:385] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.556746 23512 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.556481 23511 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.557376 23512 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.559444 23512 ts_tablet_manager.cc:1434] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:52.560326 23512 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.567860 23511 ts_tablet_manager.cc:1434] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:12:52.568288 23512 tablet_bootstrap.cc:654] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.569309 23216 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.572530 23573 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.573004 23573 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.575035 23573 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.576601 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.576627 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:52.578190 23362 leader_election.cc:343] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:52.578816 23360 leader_election.cc:343] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.579237 23360 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.579597 23510 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.580037 23510 ts_tablet_manager.cc:1403] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.101s	user 0.012s	sys 0.006s
I20260709 12:12:52.580248 23573 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.582669 23510 raft_consensus.cc:359] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.583364 23510 raft_consensus.cc:385] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.583652 23510 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.584410 23510 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.585207 23512 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.585577 23511 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.585677 23512 ts_tablet_manager.cc:1403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.026s	user 0.004s	sys 0.007s
I20260709 12:12:52.588217 23512 raft_consensus.cc:359] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.588845 23512 raft_consensus.cc:385] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.589107 23512 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.589757 23512 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.591782 23512 ts_tablet_manager.cc:1434] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:52.592068 23511 tablet_bootstrap.cc:654] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.592702 23512 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.598897 23512 tablet_bootstrap.cc:654] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.600292 23510 ts_tablet_manager.cc:1434] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:12:52.598901 23536 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.601229 23536 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.605536 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.606195 23396 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:52.607404 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:52.607863 23573 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.608048 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.608302 23620 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.608392 23573 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.609018 23620 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.609609 23620 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.611189 23573 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:52.612130 23287 leader_election.cc:343] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.612746 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:52.614825 23362 leader_election.cc:343] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.616029 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:52.617569 23360 leader_election.cc:343] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.617858 23512 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.618037 23360 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.618338 23512 ts_tablet_manager.cc:1403] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.007s
I20260709 12:12:52.619201 23573 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.621138 23512 raft_consensus.cc:359] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.621807 23512 raft_consensus.cc:385] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.622169 23512 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.623478 23620 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.626308 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.626906 23396 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.629282 23510 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.630702 23536 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.623129 23512 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.636938 23510 tablet_bootstrap.cc:654] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.637030 23396 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:52.638295 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:52.639082 23627 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.639573 23627 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.640430 23627 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.641188 23512 ts_tablet_manager.cc:1434] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:12:52.642885 23512 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
W20260709 12:12:52.647274 23287 consensus_peers.cc:597] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:52.648484 23620 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.650022 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:52.651450 23287 leader_election.cc:343] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.651891 23512 tablet_bootstrap.cc:654] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.658638 23573 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.661836 23510 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.662379 23510 ts_tablet_manager.cc:1403] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.007s
I20260709 12:12:52.665211 23510 raft_consensus.cc:359] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.666090 23510 raft_consensus.cc:385] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.666357 23510 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.667055 23510 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.666441 23216 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.668605 23510 ts_tablet_manager.cc:1434] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:52.661836 23511 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.670106 23511 ts_tablet_manager.cc:1403] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.085s	user 0.038s	sys 0.020s
I20260709 12:12:52.673048 23511 raft_consensus.cc:359] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.673693 23511 raft_consensus.cc:385] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.673965 23511 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.674679 23511 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.676848 23511 ts_tablet_manager.cc:1434] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:52.677774 23512 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.677898 23510 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.678224 23512 ts_tablet_manager.cc:1403] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.003s
I20260709 12:12:52.680871 23512 raft_consensus.cc:359] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.684098 23510 tablet_bootstrap.cc:654] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.685473 23511 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.686829 23565 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:12:52.681502 23512 raft_consensus.cc:385] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.686998 23627 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.690991 23512 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.692302 23512 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.691213 23627 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.695811 23627 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.696446 23512 ts_tablet_manager.cc:1434] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:12:52.697606 23512 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.697741 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.698340 23466 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:12:52.700271 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.701496 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.704414 23627 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:12:52.704814 23627 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.705415 23627 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.713069 23512 tablet_bootstrap.cc:654] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.714057 23627 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.716087 23540 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.716823 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.716670 23540 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.718634 23511 tablet_bootstrap.cc:654] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.720657 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.723318 23627 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:52.723253 23360 leader_election.cc:343] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.724260 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.724962 23466 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.726009 23511 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.726912 23511 ts_tablet_manager.cc:1403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.000s
I20260709 12:12:52.729266 23512 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.729733 23512 ts_tablet_manager.cc:1403] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.009s
I20260709 12:12:52.731155 23511 raft_consensus.cc:359] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.731873 23511 raft_consensus.cc:385] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.732295 23511 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.726035 23540 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.733141 23511 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.731766 23466 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:12:52.734108 23510 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.734586 23510 ts_tablet_manager.cc:1403] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.003s
I20260709 12:12:52.735215 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:52.736093 23511 ts_tablet_manager.cc:1434] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:12:52.737022 23511 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.737012 23510 raft_consensus.cc:359] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.737633 23510 raft_consensus.cc:385] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.732185 23512 raft_consensus.cc:359] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.737898 23510 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.738063 23512 raft_consensus.cc:385] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.738343 23512 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.738622 23510 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.739208 23512 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.740131 23620 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:12:52.740648 23510 ts_tablet_manager.cc:1434] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:52.740576 23620 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.741797 23620 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.743057 23510 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.744375 23603 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.747701 23512 ts_tablet_manager.cc:1434] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 12:12:52.748673 23512 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.751560 23510 tablet_bootstrap.cc:654] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.754083 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:52.755446 23362 leader_election.cc:343] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:52.755911 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.757042 23535 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.757716 23320 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:12:52.763128 23511 tablet_bootstrap.cc:654] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.765069 23320 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.766181 23512 tablet_bootstrap.cc:654] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.772893 23219 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:52.776005 23621 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:52.784263 23535 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.785039 23535 consensus_queue.cc:1243] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:12:52.785693 23540 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:12:52.786022 23540 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:52.786842 23540 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 3.1, Last appended by leader: 1, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.791718 23540 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:12:52.809885 23470 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.812856 23620 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.819377 23565 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.856444 23620 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.856951 23620 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.859848 23621 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.860378 23621 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.862620 23621 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.865039 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.865703 23511 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:52.865741 23394 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:52.866226 23511 ts_tablet_manager.cc:1403] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.129s	user 0.020s	sys 0.014s
I20260709 12:12:52.868073 23287 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:52.869663 23511 raft_consensus.cc:359] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.876552 23511 raft_consensus.cc:385] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.876410 23535 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:52.876858 23511 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.878087 23620 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.877815 23511 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.869762 23510 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:52.880303 23510 ts_tablet_manager.cc:1403] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.137s	user 0.039s	sys 0.050s
I20260709 12:12:52.870239 23628 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.881280 23628 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.881654 23628 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.884318 23510 raft_consensus.cc:359] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.885340 23510 raft_consensus.cc:385] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.872761 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.885844 23510 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.886137 23394 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:52.887176 23510 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.890190 23510 ts_tablet_manager.cc:1434] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:12:52.891489 23510 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:52.893024 23511 ts_tablet_manager.cc:1434] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 12:12:52.866322 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.898559 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:52.899245 23466 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:52.901644 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:52.902458 23511 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:52.902495 23621 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:52.903127 23621 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.903563 23621 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.893362 23628 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.910252 23511 tablet_bootstrap.cc:654] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.923103 23318 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:52.932348 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.933228 23396 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.933358 23510 tablet_bootstrap.cc:654] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.939652 23628 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:52.939767 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:52.947033 23396 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:52.951972 23512 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:52.952725 23512 ts_tablet_manager.cc:1403] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.204s	user 0.079s	sys 0.102s
I20260709 12:12:52.953402 23470 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:52.932576 23621 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.954464 23466 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:52.956528 23512 raft_consensus.cc:359] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.957530 23512 raft_consensus.cc:385] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:52.957897 23512 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:52.958851 23512 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.959569 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:52.960423 23318 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:52.962288 23287 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:52.962651 23512 ts_tablet_manager.cc:1434] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:12:52.963145 23620 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:52.963646 23620 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:52.964193 23512 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:52.964502 23620 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.968254 23621 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.969556 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:52.970382 23396 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:52.971740 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:52.973394 23524 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:52.974874 23512 tablet_bootstrap.cc:654] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:52.977836 23524 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.978350 23524 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:52.983487 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:52.984457 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:52.985110 23396 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:52.986512 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:52.989606 23621 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:52.990011 23621 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.990352 23621 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:52.991623 23620 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.996490 23524 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:52.992110 23620 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.000300 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:53.001565 23287 leader_election.cc:343] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.006050 23511 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.007068 23511 ts_tablet_manager.cc:1403] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.105s	user 0.036s	sys 0.050s
I20260709 12:12:53.009748 23217 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.010929 23620 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.010596 23511 raft_consensus.cc:359] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.011579 23511 raft_consensus.cc:385] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.011451 23620 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.012100 23511 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.013119 23511 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.016271 23511 ts_tablet_manager.cc:1434] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:12:53.020795 23621 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.021957 23466 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:53.024299 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.024781 23540 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.025761 23540 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.027858 23540 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.028281 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:53.024981 23628 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:12:53.029810 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.030015 23628 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:53.030408 23396 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.030498 23628 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:53.031636 23628 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.032153 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.034710 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:53.035130 23362 leader_election.cc:343] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.037662 23621 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.039525 23510 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.039930 23510 ts_tablet_manager.cc:1403] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.008s
I20260709 12:12:53.032932 23628 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.042007 23510 raft_consensus.cc:359] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.042654 23510 raft_consensus.cc:385] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.042938 23510 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.043577 23510 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.044691 23287 leader_election.cc:343] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.045521 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.046578 23620 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:12:53.044400 23360 leader_election.cc:343] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.047379 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.048951 23535 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.050359 23396 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:12:53.051621 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:53.052781 23621 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:53.053464 23512 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.053948 23512 ts_tablet_manager.cc:1403] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.090s	user 0.039s	sys 0.041s
I20260709 12:12:53.054034 23621 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:53.054988 23621 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.056357 23510 ts_tablet_manager.cc:1434] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:12:53.056563 23512 raft_consensus.cc:359] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.057235 23512 raft_consensus.cc:385] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.057538 23512 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.058336 23512 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.059834 23512 ts_tablet_manager.cc:1434] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:53.062192 23512 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.063743 23511 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
W20260709 12:12:53.064019 23287 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:53.068192 23510 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.069573 23512 tablet_bootstrap.cc:654] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.077277 23510 tablet_bootstrap.cc:654] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.086452 23511 tablet_bootstrap.cc:654] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.085587 23219 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:53.085631 23621 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.089905 23621 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.092619 23621 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.093194 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.093865 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:53.095114 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:53.095907 23621 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.096242 23621 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.096556 23621 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.097762 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:53.099434 23436 leader_election.cc:343] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.102614 23512 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.103063 23512 ts_tablet_manager.cc:1403] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.017s
I20260709 12:12:53.103566 23510 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.104125 23510 ts_tablet_manager.cc:1403] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.006s
I20260709 12:12:53.105705 23512 raft_consensus.cc:359] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.106357 23512 raft_consensus.cc:385] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.106611 23512 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.106946 23511 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.107100 23510 raft_consensus.cc:359] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.107440 23511 ts_tablet_manager.cc:1403] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.007s
I20260709 12:12:53.107365 23512 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.109488 23512 ts_tablet_manager.cc:1434] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:12:53.107757 23510 raft_consensus.cc:385] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.109825 23510 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.110571 23510 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.111902 23511 raft_consensus.cc:359] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.114449 23511 raft_consensus.cc:385] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.114779 23511 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.112828 23510 ts_tablet_manager.cc:1434] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:12:53.116180 23621 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.117058 23510 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.116276 23511 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.121299 23511 ts_tablet_manager.cc:1434] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.002s
I20260709 12:12:53.114050 23512 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.122879 23511 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:53.123867 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:53.124436 23321 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.126376 23540 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.126938 23512 tablet_bootstrap.cc:654] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.126828 23540 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.128854 23511 tablet_bootstrap.cc:654] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.128968 23540 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.130429 23621 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.131556 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.132189 23318 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:53.132869 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.133038 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:53.133493 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.134909 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.135625 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:12:53.136488 23362 leader_election.cc:343] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:53.136581 23436 leader_election.cc:343] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.137658 23540 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.138006 23540 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.138295 23540 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.142573 23512 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.143054 23512 ts_tablet_manager.cc:1403] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.000s
I20260709 12:12:53.144397 23540 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.145638 23512 raft_consensus.cc:359] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.146806 23512 raft_consensus.cc:385] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.147068 23512 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.147812 23512 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.148337 23510 tablet_bootstrap.cc:654] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.150954 23621 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.151469 23621 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:53.152598 23512 ts_tablet_manager.cc:1434] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:12:53.152318 23621 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.154042 23512 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.158469 23628 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.159103 23627 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.159624 23627 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.161047 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:53.162283 23318 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.164136 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.164916 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.166196 23535 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.166677 23535 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.167346 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.167505 23603 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.168030 23603 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.168715 23627 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.169126 23512 tablet_bootstrap.cc:654] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.169240 23627 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.170552 23603 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.170662 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.171612 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:53.172187 23362 leader_election.cc:343] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.172925 23535 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.173753 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.172261 23540 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.174242 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.169628 23627 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.176167 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.176652 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:53.178438 23362 leader_election.cc:343] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:53.178634 23360 leader_election.cc:343] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.180162 23360 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.181550 23535 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.182143 23466 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:12:53.176340 23627 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.184940 23436 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:12:53.187788 23287 leader_election.cc:343] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.189260 23603 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.191412 23620 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.205472 23511 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.206023 23511 ts_tablet_manager.cc:1403] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.009s
I20260709 12:12:53.208277 23627 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.210352 23627 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.231402 23510 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.231951 23510 ts_tablet_manager.cc:1403] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.115s	user 0.015s	sys 0.022s
I20260709 12:12:53.232707 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.233320 23396 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.235649 23318 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:53.239432 23396 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:53.239975 23603 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.240025 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.240520 23603 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.241217 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.242543 23603 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.241789 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.243422 23627 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.243896 23627 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:53.244665 23627 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.246451 23535 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.247803 23535 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:53.249012 23512 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.248637 23535 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.250761 23287 leader_election.cc:343] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.250519 23510 raft_consensus.cc:359] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.251359 23510 raft_consensus.cc:385] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.251607 23510 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.252202 23510 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.252219 23511 raft_consensus.cc:359] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.253835 23511 raft_consensus.cc:385] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.254096 23511 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.254868 23510 ts_tablet_manager.cc:1434] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 12:12:53.254709 23511 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.254921 23512 ts_tablet_manager.cc:1403] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.101s	user 0.020s	sys 0.008s
I20260709 12:12:53.255936 23510 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.259335 23511 ts_tablet_manager.cc:1434] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.000s
I20260709 12:12:53.261866 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.264118 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.265031 23510 tablet_bootstrap.cc:654] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.269642 23512 raft_consensus.cc:359] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.277187 23466 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.277486 23512 raft_consensus.cc:385] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.278478 23512 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.271286 23217 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.273597 23511 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:53.275734 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:53.279083 23627 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.285872 23627 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.288432 23627 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.289975 23511 tablet_bootstrap.cc:654] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.291142 23510 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.291615 23510 ts_tablet_manager.cc:1403] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.004s
I20260709 12:12:53.294678 23510 raft_consensus.cc:359] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.295346 23510 raft_consensus.cc:385] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.295619 23510 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.296264 23510 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.296787 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.297595 23394 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.271971 23216 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:53.298373 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.280167 23512 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.301316 23287 leader_election.cc:343] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.302862 23510 ts_tablet_manager.cc:1434] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:12:53.304373 23396 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:12:53.310926 23510 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.321725 23540 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.322280 23540 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.330610 23512 ts_tablet_manager.cc:1434] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 12:12:53.332082 23512 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.333488 23540 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.323621 23603 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.336907 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.343421 23536 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.343926 23536 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.344332 23536 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.334628 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.348431 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.349156 23628 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.349995 23628 consensus_queue.cc:1243] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 3.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
W20260709 12:12:53.354203 23360 leader_election.cc:343] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.354578 23573 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.355150 23573 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.358655 23362 leader_election.cc:343] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.360767 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.361632 23540 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.386317 23511 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.386874 23511 ts_tablet_manager.cc:1403] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.114s	user 0.008s	sys 0.028s
I20260709 12:12:53.387040 23510 tablet_bootstrap.cc:654] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.393853 23535 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.396502 23535 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.393610 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:53.402016 23536 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.402766 23511 raft_consensus.cc:359] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.404948 23511 raft_consensus.cc:385] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.405225 23511 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.403332 23573 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.407378 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:53.408735 23360 leader_election.cc:343] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.411581 23396 pending_rounds.cc:85] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 0
I20260709 12:12:53.412525 23396 pending_rounds.cc:107] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 3.1
I20260709 12:12:53.412818 23396 raft_consensus.cc:2889] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:53.406001 23511 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.419651 23512 tablet_bootstrap.cc:654] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.420384 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.425292 23535 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:53.426381 23360 leader_election.cc:343] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.427623 23511 ts_tablet_manager.cc:1434] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.040s	user 0.001s	sys 0.007s
I20260709 12:12:53.428025 23219 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.428933 23511 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:53.431751 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.435352 23536 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.439603 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.439824 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.440196 23394 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.440335 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:53.441146 23362 leader_election.cc:343] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.441599 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.442286 23621 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:53.442687 23362 leader_election.cc:343] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.443105 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.442883 23621 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.443835 23535 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.444621 23573 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.445645 23573 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.445957 23621 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:12:53.445314 23287 leader_election.cc:343] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.446133 23573 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.447290 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.447690 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.448201 23573 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.457867 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:53.459316 23360 leader_election.cc:343] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.461942 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:53.463390 23621 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.463380 23394 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:53.465953 23321 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:12:53.466310 23321 pending_rounds.cc:85] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 0
I20260709 12:12:53.466529 23321 pending_rounds.cc:107] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:12:53.466754 23321 raft_consensus.cc:2889] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:53.467743 23512 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.466993 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.468211 23512 ts_tablet_manager.cc:1403] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.136s	user 0.013s	sys 0.002s
I20260709 12:12:53.468921 23627 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.469540 23627 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:53.454869 23536 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.470582 23627 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.471276 23536 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.479081 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.480079 23621 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.481365 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.482651 23512 raft_consensus.cc:359] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.483328 23512 raft_consensus.cc:385] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.482053 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.476459 23536 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.483975 23394 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.483582 23512 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.485409 23512 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.488013 23362 leader_election.cc:343] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.489806 23362 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.490526 23535 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.491588 23510 tablet_bootstrap.cc:492] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
W20260709 12:12:53.487151 23287 leader_election.cc:343] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.492075 23510 ts_tablet_manager.cc:1403] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.181s	user 0.025s	sys 0.079s
I20260709 12:12:53.492565 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.494825 23510 raft_consensus.cc:359] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.495538 23510 raft_consensus.cc:385] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.495745 23512 ts_tablet_manager.cc:1434] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 12:12:53.495865 23510 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.496704 23512 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.496636 23510 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.497941 23628 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.498309 23628 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.498636 23628 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.499150 23510 ts_tablet_manager.cc:1434] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:53.500118 23510 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.504424 23512 tablet_bootstrap.cc:654] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.505808 23510 tablet_bootstrap.cc:654] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.507560 23536 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.509732 23620 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.510207 23620 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.511456 23394 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:12:53.512252 23620 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.513938 23603 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.514912 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.515534 23394 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.519445 23628 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.521310 23628 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.525636 23216 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.496783 23511 tablet_bootstrap.cc:654] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.530170 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.531490 23510 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.531999 23510 ts_tablet_manager.cc:1403] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.011s
I20260709 12:12:53.533841 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.534600 23510 raft_consensus.cc:359] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.535284 23510 raft_consensus.cc:385] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.535588 23510 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.536953 23627 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.536872 23510 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.537346 23627 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.539122 23510 ts_tablet_manager.cc:1434] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:53.539057 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.543569 23394 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.544049 23510 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.539327 23627 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.551026 23510 tablet_bootstrap.cc:654] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.555938 23510 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.556355 23510 ts_tablet_manager.cc:1403] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:12:53.558748 23510 raft_consensus.cc:359] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.559399 23510 raft_consensus.cc:385] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.559676 23510 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.561574 23510 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.563534 23510 ts_tablet_manager.cc:1434] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:12:53.564852 23510 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.569911 23510 tablet_bootstrap.cc:654] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:53.574661 23287 leader_election.cc:343] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.574928 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.581308 23510 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.580405 23394 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:53.581823 23510 ts_tablet_manager.cc:1403] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 12:12:53.584388 23510 raft_consensus.cc:359] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.585019 23510 raft_consensus.cc:385] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.585292 23510 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.585930 23510 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.588056 23620 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.588229 23627 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.589156 23511 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.589639 23511 ts_tablet_manager.cc:1403] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.161s	user 0.011s	sys 0.001s
I20260709 12:12:53.591974 23510 ts_tablet_manager.cc:1434] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:12:53.592128 23394 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.592867 23510 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.586778 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.595973 23620 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.596495 23620 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:53.599129 23620 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.599835 23510 tablet_bootstrap.cc:654] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.602521 23511 raft_consensus.cc:359] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.606590 23511 raft_consensus.cc:385] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.607188 23511 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
W20260709 12:12:53.606689 23287 leader_election.cc:343] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.606575 23603 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:53.609079 23627 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.615052 23466 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.614952 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.621773 23395 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.627784 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.629390 23603 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.638950 23395 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:53.644176 23535 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.644767 23535 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.647173 23535 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.648352 23628 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.648909 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.648870 23628 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.650923 23628 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.651518 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:53.653740 23362 leader_election.cc:343] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:53.655303 23360 leader_election.cc:343] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.655875 23360 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.659575 23535 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.667037 23511 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.662566 23216 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.677034 23511 ts_tablet_manager.cc:1434] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.087s	user 0.004s	sys 0.001s
I20260709 12:12:53.673410 23510 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.677711 23510 ts_tablet_manager.cc:1403] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.085s	user 0.026s	sys 0.015s
I20260709 12:12:53.674628 23512 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.678274 23512 ts_tablet_manager.cc:1403] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.182s	user 0.017s	sys 0.024s
I20260709 12:12:53.681345 23511 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
W20260709 12:12:53.669464 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52965.9us
I20260709 12:12:53.687227 23524 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.687767 23524 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.689910 23524 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.691601 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.692379 23717 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.692281 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.692988 23470 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.693130 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.693773 23393 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.694736 23717 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:53.694768 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.695502 23395 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:53.695983 23628 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.696108 23717 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.696645 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:53.699764 23510 raft_consensus.cc:359] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.700733 23510 raft_consensus.cc:385] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.701100 23510 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.701939 23510 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.701684 23524 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.705076 23510 ts_tablet_manager.cc:1434] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.002s
I20260709 12:12:53.705271 23524 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.705710 23524 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.705955 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.706643 23318 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
W20260709 12:12:53.707859 23287 leader_election.cc:343] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.680773 23512 raft_consensus.cc:359] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.708593 23512 raft_consensus.cc:385] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.708943 23512 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.709532 23287 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:53.709794 23512 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.710837 23628 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.711200 23628 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.711675 23628 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.712769 23512 ts_tablet_manager.cc:1434] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.002s
I20260709 12:12:53.715338 23535 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:12:53.715869 23535 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.717993 23535 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.718925 23510 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.720688 23524 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.722108 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.722749 23318 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:12:53.723054 23466 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.725283 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.726116 23470 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:53.727298 23628 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.728538 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.727334 23510 tablet_bootstrap.cc:654] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.730283 23573 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:53.730729 23573 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:12:53.731083 23573 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:53.732862 23524 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.734503 23628 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.737185 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:53.738042 23321 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:53.739406 23470 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.740051 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.740844 23467 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:53.741279 23627 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.741766 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.742421 23394 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.744724 23510 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.745186 23510 ts_tablet_manager.cc:1403] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.001s
I20260709 12:12:53.741792 23627 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.748075 23510 raft_consensus.cc:359] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.747808 23627 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.748762 23510 raft_consensus.cc:385] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.749061 23510 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.749781 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.750414 23510 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.751289 23287 leader_election.cc:343] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.752817 23512 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.753381 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.753822 23510 ts_tablet_manager.cc:1434] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:12:53.755110 23510 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.759481 23287 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.760269 23511 tablet_bootstrap.cc:654] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.759478 23603 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.761718 23536 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.761499 23603 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.764413 23394 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:53.764709 23573 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.770701 23573 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.770792 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.770424 23216 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:53.774617 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.777048 23603 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:53.777345 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.777493 23620 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:53.777976 23466 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:53.778904 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:12:53.789839 23287 leader_election.cc:343] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.790267 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.793279 23510 tablet_bootstrap.cc:654] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.797394 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:12:53.798403 23362 consensus_peers.cc:597] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:53.798576 23318 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:12:53.797420 23628 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.800971 23603 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:53.807902 23603 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:53.808815 23603 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.811980 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.811980 23512 tablet_bootstrap.cc:654] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.800990 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:53.819383 23511 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:53.833055 23511 ts_tablet_manager.cc:1403] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.155s	user 0.019s	sys 0.005s
I20260709 12:12:53.830157 23724 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:12:53.815155 23573 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:53.840382 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:53.842530 23573 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:53.861183 23524 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:53.865518 23511 raft_consensus.cc:359] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.867897 23511 raft_consensus.cc:385] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.868292 23511 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.867108 23512 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:53.869472 23512 ts_tablet_manager.cc:1403] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.156s	user 0.015s	sys 0.000s
I20260709 12:12:53.869886 23511 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.873023 23511 ts_tablet_manager.cc:1434] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 12:12:53.874365 23511 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:53.880604 23573 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.879952 23512 raft_consensus.cc:359] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.881868 23512 raft_consensus.cc:385] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.882210 23512 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.891836 23512 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.894608 23512 ts_tablet_manager.cc:1434] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 12:12:53.895628 23512 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:53.898056 23510 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:53.898563 23510 ts_tablet_manager.cc:1403] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.144s	user 0.027s	sys 0.013s
I20260709 12:12:53.900033 23511 tablet_bootstrap.cc:654] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.901564 23512 tablet_bootstrap.cc:654] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:53.904692 23216 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:53.912856 23535 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.920760 23535 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.924726 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.925691 23318 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:53.925879 23510 raft_consensus.cc:359] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.926581 23510 raft_consensus.cc:385] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:53.926839 23510 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:53.927479 23360 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:53.928774 23573 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.929273 23573 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.929749 23573 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.927497 23510 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.938985 23627 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.939765 23627 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.944867 23573 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.950181 23627 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.950790 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:53.951489 23318 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.953181 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.954106 23466 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.955160 23565 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.955834 23565 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.958312 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:53.959786 23628 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:53.960053 23535 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.960234 23628 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.960597 23628 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.960873 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:53.972564 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:53.962338 23362 leader_election.cc:343] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:53.977453 23362 leader_election.cc:343] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.978423 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.978904 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:53.979022 23628 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:53.980284 23362 leader_election.cc:343] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.980724 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:53.981727 23360 leader_election.cc:343] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:53.982149 23360 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:53.983248 23510 ts_tablet_manager.cc:1434] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 12:12:53.981419 23394 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:53.984314 23510 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:53.981200 23628 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.983054 23724 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:53.993290 23627 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.993996 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:53.994799 23470 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.997171 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:53.998276 23394 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:53.998821 23627 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:53.982277 23565 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:53.982321 23573 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.011526 23216 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.013132 23627 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.014422 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.015071 23395 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:54.015588 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:54.017086 23287 leader_election.cc:343] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.017277 23512 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.017786 23512 ts_tablet_manager.cc:1403] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.021s
I20260709 12:12:54.017877 23287 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.019129 23511 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.019397 23394 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:54.020424 23628 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.019567 23511 ts_tablet_manager.cc:1403] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.145s	user 0.057s	sys 0.035s
I20260709 12:12:54.020855 23628 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.021332 23628 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.010886 23318 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:54.022863 23360 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.023411 23511 raft_consensus.cc:359] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.023896 23565 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.024073 23511 raft_consensus.cc:385] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.024431 23511 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.024443 23565 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:54.025098 23511 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.025537 23565 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.027323 23511 ts_tablet_manager.cc:1434] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:12:54.017400 23470 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:54.028403 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:12:54.029639 23287 consensus_peers.cc:597] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:54.030920 23627 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.031957 23627 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:54.032152 23628 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.033402 23627 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.021083 23512 raft_consensus.cc:359] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.034413 23512 raft_consensus.cc:385] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.034747 23512 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.035773 23628 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.038751 23510 tablet_bootstrap.cc:654] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.037242 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.039850 23394 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.038283 23512 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.042233 23511 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.041927 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:54.043001 23216 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:54.043735 23287 leader_election.cc:343] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.044871 23512 ts_tablet_manager.cc:1434] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:12:54.045790 23512 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.051542 23512 tablet_bootstrap.cc:654] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.052986 23511 tablet_bootstrap.cc:654] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.059741 23466 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.064232 23573 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.064844 23573 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.064384 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.067137 23573 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.071177 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:54.073617 23362 leader_election.cc:343] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.076504 23717 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:54.078894 23511 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.079409 23511 ts_tablet_manager.cc:1403] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.005s
I20260709 12:12:54.081822 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:54.083456 23360 leader_election.cc:343] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.084005 23360 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.085014 23511 raft_consensus.cc:359] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.085196 23620 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.086462 23535 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.086993 23565 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.090937 23511 raft_consensus.cc:385] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.091317 23511 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.092105 23511 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.094408 23511 ts_tablet_manager.cc:1434] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:12:54.095454 23511 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.094875 23565 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.100754 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.101086 23394 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
W20260709 12:12:54.102123 23360 leader_election.cc:343] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.102815 23287 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.103925 23717 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.104529 23717 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:54.105559 23717 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.111007 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:54.112962 23362 leader_election.cc:343] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.113718 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.109779 23511 tablet_bootstrap.cc:654] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.118680 23573 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.128572 23511 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.129125 23511 ts_tablet_manager.cc:1403] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.002s
I20260709 12:12:54.135572 23511 raft_consensus.cc:359] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.136365 23511 raft_consensus.cc:385] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.136663 23511 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.137411 23511 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.144515 23511 ts_tablet_manager.cc:1434] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 12:12:54.149006 23512 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.149521 23512 ts_tablet_manager.cc:1403] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.104s	user 0.038s	sys 0.047s
I20260709 12:12:54.143086 23565 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.158728 23511 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.161405 23512 raft_consensus.cc:359] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.163079 23512 raft_consensus.cc:385] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.163427 23510 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.163497 23512 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.164037 23510 ts_tablet_manager.cc:1403] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.180s	user 0.033s	sys 0.024s
I20260709 12:12:54.164341 23512 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.174928 23512 ts_tablet_manager.cc:1434] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 12:12:54.174791 23510 raft_consensus.cc:359] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.175644 23510 raft_consensus.cc:385] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.175940 23510 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.176023 23512 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.177646 23511 tablet_bootstrap.cc:654] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.179504 23628 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.180052 23510 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.182549 23510 ts_tablet_manager.cc:1434] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:12:54.183578 23510 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:54.183672 23512 tablet_bootstrap.cc:654] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.189033 23218 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.195657 23510 tablet_bootstrap.cc:654] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.197897 23512 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.198415 23512 ts_tablet_manager.cc:1403] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.007s
I20260709 12:12:54.200618 23511 tablet_bootstrap.cc:492] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.201058 23511 ts_tablet_manager.cc:1403] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.043s	user 0.012s	sys 0.004s
I20260709 12:12:54.201174 23512 raft_consensus.cc:359] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.201882 23512 raft_consensus.cc:385] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.202144 23512 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.202982 23512 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.203619 23511 raft_consensus.cc:359] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.204393 23511 raft_consensus.cc:385] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.204644 23511 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.205075 23512 ts_tablet_manager.cc:1434] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:54.205260 23511 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.206396 23512 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.207855 23511 ts_tablet_manager.cc:1434] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:54.208775 23511 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.213364 23512 tablet_bootstrap.cc:654] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.214148 23628 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.214608 23628 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.215691 23511 tablet_bootstrap.cc:654] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.217252 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.217903 23394 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:54.219620 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:54.220463 23627 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.220911 23627 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.222846 23627 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.223073 23510 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.223330 23627 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.223716 23627 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.224239 23627 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.223711 23510 ts_tablet_manager.cc:1403] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260709 12:12:54.231360 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:54.235973 23510 raft_consensus.cc:359] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.236789 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.237030 23510 raft_consensus.cc:385] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.237489 23394 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:54.242398 23628 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.237504 23510 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
W20260709 12:12:54.244611 23287 leader_election.cc:343] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.245661 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.245383 23510 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.248493 23620 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:54.249800 23620 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.250156 23620 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.250491 23620 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.252074 23535 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.252718 23535 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.254557 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:54.255560 23510 ts_tablet_manager.cc:1434] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.002s
I20260709 12:12:54.256354 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:54.257932 23360 leader_election.cc:343] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.258352 23511 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.258832 23511 ts_tablet_manager.cc:1403] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 12:12:54.258996 23535 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.256606 23510 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
W20260709 12:12:54.260161 23287 leader_election.cc:343] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:54.261340 23287 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:54.263372 23603 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.264082 23603 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.242439 23627 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.273262 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.273415 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:54.275190 23362 leader_election.cc:343] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.275641 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.275952 23603 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.276726 23573 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.276997 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.280092 23434 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:54.281399 23511 raft_consensus.cc:359] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.283133 23603 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.288625 23627 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.289530 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.293080 23396 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.291806 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:54.299289 23396 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
W20260709 12:12:54.299808 23287 leader_election.cc:343] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.303339 23511 raft_consensus.cc:385] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.304641 23511 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.306704 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.307765 23628 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.306725 23511 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.308445 23628 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:54.315092 23628 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.303453 23620 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.324422 23510 tablet_bootstrap.cc:654] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.324656 23511 ts_tablet_manager.cc:1434] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.066s	user 0.012s	sys 0.000s
I20260709 12:12:54.325603 23511 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.326822 23512 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.327656 23620 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.328873 23512 ts_tablet_manager.cc:1403] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.123s	user 0.048s	sys 0.034s
I20260709 12:12:54.330564 23621 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.334131 23621 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.336151 23621 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.331151 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:54.333235 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.338711 23396 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.339325 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.342722 23511 tablet_bootstrap.cc:654] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:54.345460 23287 leader_election.cc:343] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.350920 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.351053 23466 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.345039 23512 raft_consensus.cc:359] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.351943 23512 raft_consensus.cc:385] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.352232 23512 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.352800 23512 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.355101 23512 ts_tablet_manager.cc:1434] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.003s
I20260709 12:12:54.355990 23512 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.359227 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:54.360047 23621 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.366415 23512 tablet_bootstrap.cc:654] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.370340 23468 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:54.371881 23565 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.373088 23396 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:54.374634 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.375715 23628 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.378381 23628 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:54.384877 23603 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:54.380051 23628 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.399290 23621 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:54.424220 23321 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:54.426249 23573 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:54.430962 23512 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.431500 23512 ts_tablet_manager.cc:1403] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.007s
I20260709 12:12:54.433203 23510 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.433998 23510 ts_tablet_manager.cc:1403] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.178s	user 0.015s	sys 0.032s
I20260709 12:12:54.434473 23512 raft_consensus.cc:359] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.435184 23512 raft_consensus.cc:385] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.435479 23512 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.437538 23510 raft_consensus.cc:359] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.440291 23510 raft_consensus.cc:385] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.440657 23510 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.441372 23510 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.437904 23512 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.443572 23510 ts_tablet_manager.cc:1434] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:12:54.444124 23218 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.450534 23510 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:54.450969 23512 ts_tablet_manager.cc:1434] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 12:12:54.451975 23512 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.452320 23536 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.452787 23536 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.454795 23536 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.457739 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:54.457913 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:54.459728 23287 leader_election.cc:343] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:54.460667 23287 leader_election.cc:343] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.461208 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.463115 23628 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.464743 23510 tablet_bootstrap.cc:654] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.475951 23511 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.476534 23511 ts_tablet_manager.cc:1403] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.151s	user 0.017s	sys 0.043s
I20260709 12:12:54.484445 23512 tablet_bootstrap.cc:654] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.479496 23511 raft_consensus.cc:359] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.492080 23511 raft_consensus.cc:385] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.498870 23511 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.500496 23511 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.503777 23511 ts_tablet_manager.cc:1434] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 12:12:54.505026 23511 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.524650 23511 tablet_bootstrap.cc:654] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.525882 23216 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:54.526662 23510 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.527235 23510 ts_tablet_manager.cc:1403] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.077s	user 0.027s	sys 0.029s
I20260709 12:12:54.533483 23512 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:54.534050 23512 ts_tablet_manager.cc:1403] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.082s	user 0.007s	sys 0.022s
I20260709 12:12:54.533869 23510 raft_consensus.cc:359] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.534561 23510 raft_consensus.cc:385] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.534821 23510 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.535421 23510 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.539517 23510 ts_tablet_manager.cc:1434] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
W20260709 12:12:54.540287 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:54.540665 23468 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.542140 23511 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.542483 23511 ts_tablet_manager.cc:1403] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.003s
I20260709 12:12:54.544740 23511 raft_consensus.cc:359] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.545231 23511 raft_consensus.cc:385] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.545406 23511 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.546272 23510 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:54.546213 23511 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.548283 23621 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.548743 23511 ts_tablet_manager.cc:1434] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:12:54.548707 23621 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.538146 23536 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.549672 23536 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.550065 23511 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.550791 23621 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.551595 23536 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.552294 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.553380 23510 tablet_bootstrap.cc:654] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.554297 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:54.558064 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.556950 23511 tablet_bootstrap.cc:654] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:54.560120 23287 leader_election.cc:343] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.560824 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.561383 23396 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:54.562291 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:54.563542 23603 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:54.564145 23603 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.564909 23603 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.565423 23510 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.566119 23510 ts_tablet_manager.cc:1403] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.012s
I20260709 12:12:54.566905 23603 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.570013 23510 raft_consensus.cc:359] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.571040 23510 raft_consensus.cc:385] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.571429 23510 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.572362 23510 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.576483 23511 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.577006 23511 ts_tablet_manager.cc:1403] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.007s
I20260709 12:12:54.577731 23510 ts_tablet_manager.cc:1434] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:12:54.579066 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.579991 23511 raft_consensus.cc:359] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.580713 23511 raft_consensus.cc:385] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.581009 23511 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.581696 23511 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.582495 23620 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.582875 23620 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.583246 23620 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.584251 23745 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.584780 23745 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.586180 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.589850 23510 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:54.591614 23511 ts_tablet_manager.cc:1434] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 12:12:54.592586 23511 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.594576 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.595577 23321 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:54.596324 23745 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.599740 23512 raft_consensus.cc:359] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.600006 23573 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.600873 23512 raft_consensus.cc:385] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.601284 23512 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.602398 23512 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.604234 23745 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.604715 23745 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.607012 23745 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.607260 23510 tablet_bootstrap.cc:654] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.609208 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.609825 23396 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:54.610232 23512 ts_tablet_manager.cc:1434] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.076s	user 0.006s	sys 0.001s
I20260709 12:12:54.614060 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:54.614702 23396 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:54.617058 23512 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:54.618018 23511 tablet_bootstrap.cc:654] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.620239 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.622099 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:12:54.624048 23434 leader_election.cc:343] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.625301 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:54.626153 23318 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:54.612154 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:54.639010 23745 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.639405 23745 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.639765 23745 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.642027 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:54.642843 23465 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.643132 23603 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.646942 23603 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.649174 23603 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.647879 23565 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:12:54.654842 23745 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.657603 23536 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:54.660815 23620 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.665094 23511 tablet_bootstrap.cc:492] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:54.665710 23511 ts_tablet_manager.cc:1403] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.073s	user 0.008s	sys 0.004s
I20260709 12:12:54.671940 23573 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.672405 23573 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.675305 23510 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:54.675412 23573 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.676946 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:54.677525 23510 ts_tablet_manager.cc:1403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.088s	user 0.020s	sys 0.007s
I20260709 12:12:54.676841 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:54.682161 23362 leader_election.cc:343] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:54.683190 23320 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:54.683301 23745 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.684208 23620 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.685667 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:54.686436 23573 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:54.686755 23573 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.687268 23573 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.684211 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.691432 23395 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.691634 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.692462 23394 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.705072 23512 tablet_bootstrap.cc:654] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:54.713977 23573 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.708772 23621 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:54.727219 23394 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:54.728809 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:54.806083 23603 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.822099 23510 raft_consensus.cc:359] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.823938 23510 raft_consensus.cc:385] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.824190 23510 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.825384 23510 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.824378 23395 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:54.830440 23511 raft_consensus.cc:359] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.831303 23511 raft_consensus.cc:385] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:54.831600 23511 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:54.831573 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:54.832428 23511 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.830860 23603 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:54.833480 23466 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.832150 23627 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:12:54.842954 23510 ts_tablet_manager.cc:1434] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.165s	user 0.008s	sys 0.000s
I20260709 12:12:54.843490 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:54.844089 23321 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.828409 23745 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:54.846299 23573 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:54.847158 23510 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:54.849291 23621 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:12:54.849761 23621 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:54.850841 23466 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:54.826915 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:54.838960 23470 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:54.870157 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 141455us
I20260709 12:12:54.876592 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:54.877189 23394 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.887115 23394 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
W20260709 12:12:54.948124 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 123347us
I20260709 12:12:54.873222 23320 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:54.964043 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:54.973591 23565 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.974628 23565 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:54.976063 23511 ts_tablet_manager.cc:1434] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.310s	user 0.032s	sys 0.055s
I20260709 12:12:54.978789 23511 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:54.888321 23745 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:54.913196 23621 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:55.022436 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58901.2us
I20260709 12:12:55.024638 23627 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.031553 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:55.029779 23321 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:55.025172 23621 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.040194 23524 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.025602 23627 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.063673 23783 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.070031 23745 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:55.067441 23362 leader_election.cc:343] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.072352 23468 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.073397 23565 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.086619 23524 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:55.092669 23779 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.101418 23627 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.102715 23465 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:12:55.095821 23779 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.091060 23783 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.108121 23783 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.109704 23786 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.087989 23524 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.110458 23786 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.111908 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.114379 23786 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.115656 23512 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.117694 23512 ts_tablet_manager.cc:1403] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.502s	user 0.020s	sys 0.021s
I20260709 12:12:55.119122 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.120056 23320 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:55.119426 23511 tablet_bootstrap.cc:654] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:55.106925 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94925.5us
I20260709 12:12:55.131353 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.132755 23783 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.133872 23783 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.209997 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.212194 23510 tablet_bootstrap.cc:654] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.212280 23789 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.213513 23783 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.213616 23789 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:55.218706 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:12:55.220276 23362 leader_election.cc:343] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.222793 23512 raft_consensus.cc:359] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.224802 23512 raft_consensus.cc:385] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.225821 23512 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.226819 23512 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.227387 23779 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.233996 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.215237 23789 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.241721 23512 ts_tablet_manager.cc:1434] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.018s
I20260709 12:12:55.241631 23787 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:55.242430 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:55.243209 23512 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:55.240558 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.238775 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:12:55.271001 23287 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:55.272094 23394 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:55.275858 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.277130 23318 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:55.280112 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:55.281399 23779 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:55.282269 23779 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.282681 23779 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:55.281522 23627 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:55.274842 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.286549 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:55.291415 23360 leader_election.cc:343] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.291932 23360 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.301023 23775 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.301487 23775 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.301831 23775 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.284197 23790 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.148s
I20260709 12:12:55.306959 23784 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
W20260709 12:12:55.307832 23360 leader_election.cc:343] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.315896 23779 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.325131 23218 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:55.328173 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.334172 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.338162 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.331215 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.341964 23393 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:55.318682 23779 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.348590 23511 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:55.348965 23511 ts_tablet_manager.cc:1403] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.371s	user 0.023s	sys 0.049s
I20260709 12:12:55.356536 23786 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:55.357075 23786 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:55.353946 23628 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.364614 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.352972 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:55.365801 23362 leader_election.cc:343] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:55.370750 23434 leader_election.cc:343] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.371708 23362 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.368114 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.372550 23783 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.370003 23788 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260709 12:12:55.384344 23287 consensus_peers.cc:597] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:12:55.385733 23287 leader_election.cc:343] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.370162 23628 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:55.387118 23628 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.379943 23775 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:55.390623 23362 leader_election.cc:343] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.394650 23511 raft_consensus.cc:359] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.395558 23511 raft_consensus.cc:385] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.395895 23511 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.396901 23511 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.400969 23511 ts_tablet_manager.cc:1434] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.052s	user 0.001s	sys 0.005s
I20260709 12:12:55.372885 23786 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.406639 23786 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:55.408242 23510 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:55.408492 23511 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:55.411669 23510 ts_tablet_manager.cc:1403] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.565s	user 0.018s	sys 0.031s
W20260709 12:12:55.417294 23360 leader_election.cc:343] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.417910 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.431838 23512 tablet_bootstrap.cc:654] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.423175 23783 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:12:55.435825 23791 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.144s	user 0.004s	sys 0.001s
W20260709 12:12:55.517627 23794 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.070s	user 0.003s	sys 0.000s
I20260709 12:12:55.518203 23779 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.518858 23779 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.524595 23217 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:55.527524 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 110290us
I20260709 12:12:55.432957 23775 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.528378 23779 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:12:55.498898 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 60683.1us
I20260709 12:12:55.434371 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:55.532493 23510 raft_consensus.cc:359] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:55.535576 23362 leader_election.cc:343] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.536793 23510 raft_consensus.cc:385] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:12:55.537389 23773 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.098s	user 0.000s	sys 0.018s
I20260709 12:12:55.537899 23510 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
W20260709 12:12:55.538527 23792 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.143s	user 0.000s	sys 0.003s
W20260709 12:12:55.539428 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 112392us
I20260709 12:12:55.539778 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.541584 23314 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.540061 23510 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.540694 23779 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:55.548435 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.549685 23788 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.550159 23788 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.550580 23788 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.538817 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.556070 23314 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:55.558825 23434 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:55.560719 23786 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:55.562332 23510 ts_tablet_manager.cc:1434] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.150s	user 0.009s	sys 0.007s
I20260709 12:12:55.567546 23779 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:12:55.577620 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:55.577935 23360 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:55.579363 23790 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.581184 23790 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:55.582485 23790 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.586716 23784 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.592748 23512 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.593216 23512 ts_tablet_manager.cc:1403] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.350s	user 0.015s	sys 0.019s
I20260709 12:12:55.591666 23784 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.598652 23784 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.598804 23788 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.600636 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.600944 23788 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.604290 23510 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:55.601377 23395 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:55.606108 23511 tablet_bootstrap.cc:654] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.607754 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:12:55.616320 23362 leader_election.cc:343] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.615283 23512 raft_consensus.cc:359] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.617872 23512 raft_consensus.cc:385] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.618158 23779 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.618274 23512 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.618611 23779 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.619005 23779 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.619295 23512 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.622673 23512 ts_tablet_manager.cc:1434] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 12:12:55.624291 23512 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:55.629879 23467 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:12:55.637280 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.638837 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:55.641312 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.626444 23779 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.644130 23745 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:55.645674 23779 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.647193 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.647835 23393 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:55.655223 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.655436 23512 tablet_bootstrap.cc:654] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.655817 23393 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.659405 23784 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:55.660202 23784 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:12:55.660526 23784 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:55.661343 23784 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.662698 23784 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:55.662909 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.663504 23394 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.665033 23510 tablet_bootstrap.cc:654] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.673804 23511 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:55.674253 23511 ts_tablet_manager.cc:1403] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.266s	user 0.013s	sys 0.002s
I20260709 12:12:55.674090 23774 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:55.674628 23287 leader_election.cc:343] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:55.675303 23287 leader_election.cc:343] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.678121 23510 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:55.678560 23510 ts_tablet_manager.cc:1403] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.001s
I20260709 12:12:55.681818 23511 raft_consensus.cc:359] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.682685 23511 raft_consensus.cc:385] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.683014 23511 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
W20260709 12:12:55.682878 23434 leader_election.cc:343] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.683813 23511 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.685503 23394 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:55.685326 23510 raft_consensus.cc:359] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.688628 23510 raft_consensus.cc:385] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.688752 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.688951 23510 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.689807 23510 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.690668 23788 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.691063 23788 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.691412 23788 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.691552 23512 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.692170 23512 ts_tablet_manager.cc:1403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.068s	user 0.006s	sys 0.007s
I20260709 12:12:55.693420 23510 ts_tablet_manager.cc:1434] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:12:55.694898 23510 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:55.696935 23512 raft_consensus.cc:359] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.698225 23779 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:55.698974 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:55.698902 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.699241 23511 ts_tablet_manager.cc:1434] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.003s
I20260709 12:12:55.701608 23511 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:55.702816 23510 tablet_bootstrap.cc:654] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.707986 23511 tablet_bootstrap.cc:654] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
W20260709 12:12:55.709486 23434 leader_election.cc:343] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.709949 23512 raft_consensus.cc:385] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.712476 23512 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.713305 23512 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.721649 23512 ts_tablet_manager.cc:1434] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:12:55.729480 23512 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:55.730806 23745 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:55.717476 23788 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.734268 23788 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.736464 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:55.739034 23511 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:55.739558 23511 ts_tablet_manager.cc:1403] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.038s	user 0.006s	sys 0.006s
W20260709 12:12:55.743096 23287 leader_election.cc:343] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.743398 23511 raft_consensus.cc:359] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.746372 23511 raft_consensus.cc:385] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.746836 23510 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:55.747049 23745 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.747289 23510 ts_tablet_manager.cc:1403] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.034s
I20260709 12:12:55.747516 23745 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.749768 23745 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.750236 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.750429 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.751006 23390 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.746687 23511 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.752158 23784 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.752678 23784 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:55.717291 23393 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:55.752193 23511 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.753603 23784 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.758992 23511 ts_tablet_manager.cc:1434] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:12:55.759847 23390 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:55.756491 23512 tablet_bootstrap.cc:654] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.754479 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:55.761250 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.761976 23745 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.762450 23745 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:12:55.762867 23434 leader_election.cc:343] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.763265 23745 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.760084 23511 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:55.770254 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.770975 23393 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:55.779563 23510 raft_consensus.cc:359] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.780285 23510 raft_consensus.cc:385] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.780568 23510 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.780784 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:55.781760 23781 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.781405 23510 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.782168 23781 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.782548 23781 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.784569 23510 ts_tablet_manager.cc:1434] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.003s
I20260709 12:12:55.785674 23510 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:55.789103 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.790282 23717 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:12:55.790467 23362 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:55.790752 23717 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:55.791646 23717 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.793902 23781 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.796543 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.797014 23781 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:12:55.798341 23434 leader_election.cc:343] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.802560 23510 tablet_bootstrap.cc:654] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.802937 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.803467 23394 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:55.812875 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:55.817657 23510 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:55.818785 23510 ts_tablet_manager.cc:1403] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.000s
I20260709 12:12:55.819998 23394 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:55.821007 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:55.822175 23745 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.822445 23512 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.822252 23510 raft_consensus.cc:359] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.822731 23745 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:55.823001 23510 raft_consensus.cc:385] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.823091 23512 ts_tablet_manager.cc:1403] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.094s	user 0.009s	sys 0.006s
I20260709 12:12:55.823374 23510 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.823582 23745 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.824307 23510 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.826352 23512 raft_consensus.cc:359] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.826752 23510 ts_tablet_manager.cc:1434] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:12:55.827005 23512 raft_consensus.cc:385] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.827378 23512 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.827979 23510 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:55.828202 23512 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.830442 23512 ts_tablet_manager.cc:1434] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:12:55.834198 23512 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:55.836148 23510 tablet_bootstrap.cc:654] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.820554 23511 tablet_bootstrap.cc:654] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.849931 23774 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.850549 23774 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.855273 23512 tablet_bootstrap.cc:654] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.858946 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.861855 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.857223 23774 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.876569 23779 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.878295 23779 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.876828 23319 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:55.880407 23779 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.881297 23360 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:12:55.882817 23362 leader_election.cc:343] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.884178 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.885035 23394 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:55.889515 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:55.887836 23774 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.893956 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:12:55.896059 23434 leader_election.cc:343] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.895259 23774 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.897089 23774 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.900255 23512 tablet_bootstrap.cc:492] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.900727 23512 ts_tablet_manager.cc:1403] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.069s	user 0.005s	sys 0.019s
I20260709 12:12:55.900730 23784 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:55.902101 23745 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:55.902495 23745 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.902845 23745 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.903715 23512 raft_consensus.cc:359] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.911283 23511 tablet_bootstrap.cc:492] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:55.904431 23512 raft_consensus.cc:385] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.911759 23512 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.911984 23511 ts_tablet_manager.cc:1403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.152s	user 0.013s	sys 0.016s
I20260709 12:12:55.912462 23512 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.914579 23512 ts_tablet_manager.cc:1434] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.004s
I20260709 12:12:55.914644 23511 raft_consensus.cc:359] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.915395 23511 raft_consensus.cc:385] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.915624 23512 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:55.915634 23511 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.917387 23511 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.911283 23510 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:55.920372 23510 ts_tablet_manager.cc:1403] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.006s
I20260709 12:12:55.924750 23511 ts_tablet_manager.cc:1434] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:12:55.925814 23511 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:55.938277 23512 tablet_bootstrap.cc:654] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:55.938458 23717 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.939186 23717 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.950323 23786 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.951052 23786 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.959160 23510 raft_consensus.cc:359] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.959913 23510 raft_consensus.cc:385] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.960086 23786 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.960675 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:55.962427 23360 leader_election.cc:343] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.962224 23774 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.927119 23218 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:55.964016 23784 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:55.965090 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.965723 23319 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.965926 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:55.967250 23362 leader_election.cc:343] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.960202 23510 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.968225 23510 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.964583 23218 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:12:55.962698 23745 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.974248 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:55.976979 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:55.977159 23510 ts_tablet_manager.cc:1434] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.057s	user 0.013s	sys 0.002s
I20260709 12:12:55.977595 23395 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.978477 23510 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:55.978941 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:55.979555 23316 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:55.985571 23512 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:55.986132 23512 ts_tablet_manager.cc:1403] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.004s
I20260709 12:12:55.986845 23745 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:55.987856 23395 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:55.971993 23774 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:55.989097 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:12:55.991613 23362 leader_election.cc:343] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:55.992192 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.990783 23745 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:55.990237 23319 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:55.990743 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:55.988910 23512 raft_consensus.cc:359] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.993996 23512 raft_consensus.cc:385] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:55.994095 23360 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:55.994325 23512 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:55.994966 23512 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.993564 23745 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:55.999257 23745 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.019811 23217 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.029609 23783 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.030366 23783 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:55.996913 23789 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.032074 23789 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:56.034390 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.034942 23788 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.035580 23510 tablet_bootstrap.cc:654] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.035395 23788 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.036161 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.032934 23783 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.038117 23627 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.038955 23217 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:55.995144 23790 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.033154 23717 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.040095 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.042321 23217 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.044922 23511 tablet_bootstrap.cc:654] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.033047 23789 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:56.052706 23360 leader_election.cc:343] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.057529 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.057880 23512 ts_tablet_manager.cc:1434] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.071s	user 0.011s	sys 0.007s
I20260709 12:12:56.058936 23512 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
W20260709 12:12:56.060710 23362 leader_election.cc:343] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.061146 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.062402 23786 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.065876 23788 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.072355 23783 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:12:56.072459 23316 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:56.077551 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.088925 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.089788 23393 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:56.091284 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.092132 23788 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.096566 23512 tablet_bootstrap.cc:654] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.099366 23788 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.104161 23216 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.107165 23216 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.111105 23510 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.111593 23510 ts_tablet_manager.cc:1403] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.133s	user 0.016s	sys 0.004s
W20260709 12:12:56.113253 23362 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:56.114773 23510 raft_consensus.cc:359] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.115293 23510 raft_consensus.cc:385] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.115530 23510 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.116148 23510 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.118952 23510 ts_tablet_manager.cc:1434] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:12:56.119959 23510 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.121292 23218 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.128377 23789 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.128945 23789 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.132066 23774 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.132519 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.132593 23774 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.133199 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:56.134085 23362 leader_election.cc:343] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.134485 23717 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.134910 23774 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.134968 23717 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.131247 23789 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.136528 23510 tablet_bootstrap.cc:654] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.136958 23511 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.137039 23717 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.137506 23511 ts_tablet_manager.cc:1403] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.212s	user 0.020s	sys 0.007s
W20260709 12:12:56.138449 23360 leader_election.cc:343] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.138863 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.139813 23789 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.140769 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.140321 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:56.142153 23362 leader_election.cc:343] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:56.142807 23360 leader_election.cc:343] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.143239 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.144253 23789 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.148200 23511 raft_consensus.cc:359] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.148906 23511 raft_consensus.cc:385] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.149169 23511 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.149816 23511 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.151327 23511 ts_tablet_manager.cc:1434] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 12:12:56.152355 23511 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.155624 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.156455 23394 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:56.156723 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.157366 23467 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:56.165989 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:56.166761 23717 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.167111 23717 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.167460 23717 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:12:56.181856 23287 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:56.183362 23717 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.185930 23717 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.188876 23315 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.192369 23512 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.193045 23512 ts_tablet_manager.cc:1403] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.134s	user 0.020s	sys 0.019s
I20260709 12:12:56.194728 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:56.195492 23467 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.196141 23512 raft_consensus.cc:359] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.196828 23512 raft_consensus.cc:385] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.197095 23512 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.197844 23512 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.197576 23511 tablet_bootstrap.cc:654] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.200263 23512 ts_tablet_manager.cc:1434] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:56.201300 23512 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.194721 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:12:56.202781 23287 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:56.202929 23394 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.203329 23510 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.203770 23510 ts_tablet_manager.cc:1403] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.084s	user 0.035s	sys 0.025s
I20260709 12:12:56.206386 23510 raft_consensus.cc:359] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.207043 23510 raft_consensus.cc:385] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.207306 23510 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.207705 23512 tablet_bootstrap.cc:654] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.210672 23510 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.212560 23745 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:56.213227 23510 ts_tablet_manager.cc:1434] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:12:56.214359 23510 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.221066 23510 tablet_bootstrap.cc:654] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.221509 23394 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:56.226253 23467 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:56.249092 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:56.250175 23788 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.250730 23788 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:56.251724 23788 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.259176 23788 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:12:56.302043 23434 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:56.314137 23511 tablet_bootstrap.cc:492] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.314673 23511 ts_tablet_manager.cc:1403] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.163s	user 0.027s	sys 0.022s
I20260709 12:12:56.317719 23511 raft_consensus.cc:359] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.318369 23511 raft_consensus.cc:385] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.318627 23511 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.319252 23511 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.321579 23511 ts_tablet_manager.cc:1434] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:56.322738 23511 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.323485 23218 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.327003 23510 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.327461 23510 ts_tablet_manager.cc:1403] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.113s	user 0.024s	sys 0.027s
I20260709 12:12:56.328224 23512 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.328685 23512 ts_tablet_manager.cc:1403] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.128s	user 0.045s	sys 0.050s
I20260709 12:12:56.330166 23510 raft_consensus.cc:359] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.330826 23510 raft_consensus.cc:385] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.331076 23510 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.331437 23512 raft_consensus.cc:359] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.331881 23511 tablet_bootstrap.cc:654] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.332096 23512 raft_consensus.cc:385] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.332396 23512 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.335579 23717 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.336048 23717 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.331715 23510 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.339277 23510 ts_tablet_manager.cc:1434] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:12:56.340080 23512 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.342612 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.343223 23393 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:56.343567 23512 ts_tablet_manager.cc:1434] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.006s
I20260709 12:12:56.344403 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:56.344848 23512 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.345197 23788 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.345589 23788 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.345898 23788 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.346446 23627 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:12:56.346884 23627 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.347580 23510 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.348640 23627 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.347608 23717 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.355458 23467 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.359016 23510 tablet_bootstrap.cc:654] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.361270 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.362193 23512 tablet_bootstrap.cc:654] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.361995 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.362867 23465 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:56.367084 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.367690 23393 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:12:56.370115 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:12:56.371343 23287 leader_election.cc:343] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.371781 23627 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:56.372225 23627 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:12:56.372643 23627 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:12:56.382707 23745 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:56.390662 23745 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:56.391587 23511 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.392577 23511 ts_tablet_manager.cc:1403] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.070s	user 0.019s	sys 0.008s
I20260709 12:12:56.388748 23627 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.384696 23788 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.395694 23788 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.402989 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:56.403620 23390 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.404795 23627 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.397104 23216 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.408043 23511 raft_consensus.cc:359] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.409487 23511 raft_consensus.cc:385] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.410012 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:56.414219 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:56.416831 23511 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.407608 23745 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:12:56.417888 23511 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.422290 23390 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:56.428048 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:56.433951 23287 leader_election.cc:343] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.434490 23467 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:56.437373 23511 ts_tablet_manager.cc:1434] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 12:12:56.429710 23512 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.444824 23512 ts_tablet_manager.cc:1403] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.100s	user 0.025s	sys 0.015s
I20260709 12:12:56.447970 23510 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.448850 23510 ts_tablet_manager.cc:1403] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.101s	user 0.021s	sys 0.019s
I20260709 12:12:56.451596 23510 raft_consensus.cc:359] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.452296 23510 raft_consensus.cc:385] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.452607 23510 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.453392 23510 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.458009 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.458866 23627 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.459275 23627 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:56.460033 23627 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.456404 23510 ts_tablet_manager.cc:1434] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:56.445169 23393 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:12:56.465945 23511 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.473658 23717 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:56.476451 23510 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.477577 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.479015 23627 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:56.481383 23511 tablet_bootstrap.cc:654] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.484057 23627 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:56.486536 23627 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:56.496294 23362 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:56.496955 23512 raft_consensus.cc:359] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.497774 23512 raft_consensus.cc:385] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.498034 23512 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.498659 23512 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.499917 23781 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.500427 23781 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.502588 23219 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.502607 23781 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:56.504272 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.507431 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.507918 23790 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.508560 23790 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.509449 23467 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.513178 23790 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.513418 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.514168 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:56.515430 23362 leader_election.cc:343] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.516180 23512 ts_tablet_manager.cc:1434] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.002s
I20260709 12:12:56.517405 23512 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
W20260709 12:12:56.520375 23360 leader_election.cc:343] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.523777 23512 tablet_bootstrap.cc:654] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.520943 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.526796 23510 tablet_bootstrap.cc:654] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.530253 23789 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.514112 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:56.531576 23789 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.530709 23774 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.535300 23789 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.537848 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.539765 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:12:56.542140 23362 leader_election.cc:343] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.546638 23781 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:12:56.549867 23360 leader_election.cc:343] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.552309 23511 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.552837 23511 ts_tablet_manager.cc:1403] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.016s
I20260709 12:12:56.550343 23360 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.555541 23511 raft_consensus.cc:359] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.556275 23790 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.556416 23511 raft_consensus.cc:385] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.556821 23511 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.557755 23511 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.559100 23790 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:56.558550 23219 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.564883 23510 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.565423 23510 ts_tablet_manager.cc:1403] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.010s
I20260709 12:12:56.567250 23511 ts_tablet_manager.cc:1434] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 12:12:56.568248 23510 raft_consensus.cc:359] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.568923 23510 raft_consensus.cc:385] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.569181 23510 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.570065 23510 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.570850 23511 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.572268 23510 ts_tablet_manager.cc:1434] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:56.573345 23510 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.581030 23512 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.581537 23512 ts_tablet_manager.cc:1403] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.010s
I20260709 12:12:56.583907 23511 tablet_bootstrap.cc:654] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.584306 23512 raft_consensus.cc:359] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.584990 23512 raft_consensus.cc:385] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.585268 23512 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.586607 23512 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.590366 23469 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.594561 23512 ts_tablet_manager.cc:1434] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:12:56.595499 23627 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.595956 23627 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.597961 23627 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.599593 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.600193 23394 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:56.600133 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.601392 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:12:56.602201 23287 leader_election.cc:343] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.605309 23627 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.605724 23627 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.605988 23627 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.613878 23512 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.618697 23627 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.621049 23627 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.625017 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:12:56.627658 23287 leader_election.cc:343] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.635061 23717 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.635607 23717 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.626776 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:56.638362 23390 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.638819 23717 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.644222 23510 tablet_bootstrap.cc:654] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.646741 23512 tablet_bootstrap.cc:654] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.657846 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.660346 23779 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.668630 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.679414 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.680534 23787 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.688678 23511 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.689209 23511 ts_tablet_manager.cc:1403] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.119s	user 0.018s	sys 0.011s
I20260709 12:12:56.693507 23511 raft_consensus.cc:359] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.695604 23511 raft_consensus.cc:385] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.695900 23510 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.696403 23510 ts_tablet_manager.cc:1403] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.123s	user 0.010s	sys 0.002s
I20260709 12:12:56.700001 23510 raft_consensus.cc:359] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.700663 23510 raft_consensus.cc:385] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.700677 23390 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:56.695945 23511 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.701014 23510 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.702076 23511 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.702312 23510 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.703645 23511 ts_tablet_manager.cc:1434] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 12:12:56.704831 23511 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.704861 23510 ts_tablet_manager.cc:1434] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:12:56.707708 23510 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.711225 23511 tablet_bootstrap.cc:654] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.703207 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.714082 23717 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:56.714598 23717 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:56.715415 23717 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.716127 23511 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.716557 23511 ts_tablet_manager.cc:1403] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:12:56.719354 23510 tablet_bootstrap.cc:654] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.719662 23511 raft_consensus.cc:359] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.720312 23511 raft_consensus.cc:385] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.720580 23511 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.721191 23511 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.723451 23511 ts_tablet_manager.cc:1434] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:12:56.724391 23511 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.730072 23511 tablet_bootstrap.cc:654] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.744975 23511 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.745473 23511 ts_tablet_manager.cc:1403] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.002s
I20260709 12:12:56.747016 23512 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.747604 23512 ts_tablet_manager.cc:1403] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.134s	user 0.015s	sys 0.010s
I20260709 12:12:56.748862 23511 raft_consensus.cc:359] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.749548 23511 raft_consensus.cc:385] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.749800 23511 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.750588 23511 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.753357 23510 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.753989 23510 ts_tablet_manager.cc:1403] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.011s
I20260709 12:12:56.754025 23511 ts_tablet_manager.cc:1434] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:12:56.753165 23216 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.755064 23511 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.756758 23779 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.756896 23510 raft_consensus.cc:359] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.750660 23512 raft_consensus.cc:359] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.757339 23779 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.757640 23510 raft_consensus.cc:385] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.758253 23512 raft_consensus.cc:385] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.758525 23510 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.758644 23512 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.759475 23512 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.759308 23510 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.761271 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.762110 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.764093 23510 ts_tablet_manager.cc:1434] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:12:56.764568 23512 ts_tablet_manager.cc:1434] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 12:12:56.765575 23512 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.766749 23434 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:56.766897 23786 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.767604 23745 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.767800 23786 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.769809 23511 tablet_bootstrap.cc:654] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.770728 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.771327 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:56.772143 23360 leader_election.cc:343] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.772689 23779 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:12:56.773797 23362 leader_election.cc:343] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.774262 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.773602 23786 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.775437 23786 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.777016 23510 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.785264 23510 tablet_bootstrap.cc:654] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.788182 23512 tablet_bootstrap.cc:654] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.794603 23788 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:56.796134 23788 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:56.803002 23512 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.803525 23512 ts_tablet_manager.cc:1403] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 12:12:56.806365 23512 raft_consensus.cc:359] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.807017 23512 raft_consensus.cc:385] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.807286 23512 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.808432 23510 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:56.808022 23512 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.809059 23510 ts_tablet_manager.cc:1403] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.011s
I20260709 12:12:56.810329 23512 ts_tablet_manager.cc:1434] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:12:56.815610 23511 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.818861 23512 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.823549 23511 ts_tablet_manager.cc:1403] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.069s	user 0.008s	sys 0.023s
I20260709 12:12:56.824877 23512 tablet_bootstrap.cc:654] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.837800 23319 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.828409 23511 raft_consensus.cc:359] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.838840 23510 raft_consensus.cc:359] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.839618 23511 raft_consensus.cc:385] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.840090 23511 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.839788 23510 raft_consensus.cc:385] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.840729 23510 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.845795 23510 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.850927 23510 ts_tablet_manager.cc:1434] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.002s
I20260709 12:12:56.840996 23511 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.855530 23511 ts_tablet_manager.cc:1434] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.004s
I20260709 12:12:56.856734 23511 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:56.868489 23510 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Bootstrap starting.
I20260709 12:12:56.873555 23745 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.879098 23745 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.892175 23745 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:56.895190 23788 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.898789 23788 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.900844 23788 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.914357 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.918421 23511 tablet_bootstrap.cc:654] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.925189 23469 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.927577 23510 tablet_bootstrap.cc:654] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:56.934751 23774 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.917286 23390 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:56.942566 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:56.949498 23774 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.951646 23745 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.952096 23745 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.954319 23774 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.952482 23745 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.955055 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.956923 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:56.957998 23360 leader_election.cc:343] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:12:56.958645 23362 leader_election.cc:343] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:56.959818 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:56.961505 23774 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:56.961793 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:56.962805 23315 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:56.959578 23779 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.963670 23779 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.965760 23779 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:56.967248 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.967957 23390 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:56.968039 23745 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.970710 23512 tablet_bootstrap.cc:492] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:56.970559 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:56.971176 23512 ts_tablet_manager.cc:1403] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.153s	user 0.033s	sys 0.066s
I20260709 12:12:56.982216 23788 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.983004 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:56.982853 23788 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.984752 23512 raft_consensus.cc:359] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.989605 23512 raft_consensus.cc:385] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:56.990036 23512 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:56.990846 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:56.990715 23512 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.991425 23390 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:56.992904 23512 ts_tablet_manager.cc:1434] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.004s
I20260709 12:12:56.993254 23511 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:56.988564 23788 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:56.993912 23512 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:56.995090 23779 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:56.995509 23779 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.995855 23779 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.996163 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:56.996745 23396 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:56.989416 23745 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.002666 23390 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:57.007766 23396 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:57.007795 23779 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:56.993803 23511 ts_tablet_manager.cc:1403] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.137s	user 0.018s	sys 0.003s
I20260709 12:12:57.013948 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.014611 23319 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:57.015116 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:57.015652 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:57.016320 23390 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.018111 23510 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: No bootstrap required, opened a new log
I20260709 12:12:57.018563 23510 ts_tablet_manager.cc:1403] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Time spent bootstrapping tablet: real 0.150s	user 0.021s	sys 0.011s
I20260709 12:12:57.018790 23512 tablet_bootstrap.cc:654] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.020980 23510 raft_consensus.cc:359] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.021765 23510 raft_consensus.cc:385] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.022022 23510 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.022074 23745 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:57.022648 23510 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.023108 23745 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:57.024366 23788 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.024887 23788 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.026746 23788 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:12:57.037711 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:57.056794 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:57.057476 23319 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.058482 23745 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.063563 23390 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:57.064826 23319 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:57.067723 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:57.068154 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:57.068727 23314 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.068811 23855 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:57.069514 23855 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:57.070626 23855 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.075366 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:57.075580 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:57.012604 23779 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.077205 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:57.077862 23465 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:57.077986 23511 raft_consensus.cc:359] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.078686 23511 raft_consensus.cc:385] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.078955 23511 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.079982 23784 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:57.080367 23784 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.080713 23784 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.084414 23627 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:12:57.084792 23511 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:57.087019 23287 leader_election.cc:343] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.087357 23510 ts_tablet_manager.cc:1434] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.000s
I20260709 12:12:57.087565 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.088203 23395 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:57.098111 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.099112 23396 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:57.103613 23314 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
W20260709 12:12:57.122515 23287 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:57.123456 23511 ts_tablet_manager.cc:1434] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.005s
I20260709 12:12:57.126681 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:57.129602 23881 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:57.134783 23881 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.135143 23881 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.139115 23511 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:57.138234 23779 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.139885 23779 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.141984 23779 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.144300 23470 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:57.148484 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.149364 23393 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:57.143028 23790 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:12:57.149999 23790 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.136899 23855 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.153403 23287 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.152100 23790 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.156677 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.162860 23315 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:12:57.157768 23717 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:57.158353 23784 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.165491 23784 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.158761 23855 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.159749 23627 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:57.162290 23512 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:57.168979 23512 ts_tablet_manager.cc:1403] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.175s	user 0.036s	sys 0.036s
I20260709 12:12:57.171396 23360 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:57.190299 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:57.185276 23219 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:57.181816 23789 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:12:57.197158 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:57.206441 23465 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.203053 23855 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.224318 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.204639 23789 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:12:57.227684 23469 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:57.227713 23892 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:57.230259 23511 tablet_bootstrap.cc:654] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.232306 23881 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.233009 23789 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:12:57.228873 23287 leader_election.cc:343] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.237314 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.249577 23434 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:12:57.249598 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53991.6us
I20260709 12:12:57.261920 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.264367 23790 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.255635 23893 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.251659 23889 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.282647 23881 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.282832 23893 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.273527 23316 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:57.282672 23512 raft_consensus.cc:359] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.287075 23512 raft_consensus.cc:385] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.287364 23512 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.289018 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:57.290057 23470 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.295125 23512 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.272724 23790 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.303299 23470 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:57.303671 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:57.304257 23390 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.302253 23893 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.309458 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:57.314628 23362 leader_election.cc:343] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.323964 23512 ts_tablet_manager.cc:1434] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.155s	user 0.009s	sys 0.001s
I20260709 12:12:57.313895 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:57.324481 23779 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:12:57.329842 23511 tablet_bootstrap.cc:492] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:57.330338 23511 ts_tablet_manager.cc:1403] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.193s	user 0.017s	sys 0.015s
I20260709 12:12:57.329983 23779 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.332181 23395 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.356549 23511 raft_consensus.cc:359] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.357314 23511 raft_consensus.cc:385] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.357604 23511 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.358379 23511 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.362612 23390 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:57.368573 23790 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.376752 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:57.390312 23362 leader_election.cc:343] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.393085 23889 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:12:57.418285 23897 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:12:57.421340 23395 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:57.425277 23892 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:57.427924 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.428733 23892 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:57.428975 23789 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.430160 23396 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:12:57.429276 23892 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:57.432864 23892 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.434715 23789 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.436122 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:57.437067 23467 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:57.438265 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:57.440035 23627 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:57.440524 23627 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:57.441294 23627 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.456780 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:57.460752 23890 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:12:57.466991 23511 ts_tablet_manager.cc:1434] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.136s	user 0.007s	sys 0.005s
I20260709 12:12:57.468183 23511 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:57.471312 23779 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.474527 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.473320 23512 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:57.476661 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.478796 23511 tablet_bootstrap.cc:654] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.484869 23887 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:57.486277 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:57.495429 23512 tablet_bootstrap.cc:654] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.496053 23314 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
W20260709 12:12:57.499171 23360 leader_election.cc:343] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.499634 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:12:57.513053 23360 leader_election.cc:343] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.513619 23360 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.513948 23216 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:57.517192 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.519472 23774 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:57.520215 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.559247 23360 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.572170 23895 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:57.565645 23790 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:12:57.574529 23790 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:57.575029 23511 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:57.575464 23511 ts_tablet_manager.cc:1403] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.017s
I20260709 12:12:57.575449 23790 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.583141 23889 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:12:57.643911 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:57.599402 23781 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:12:57.646649 23781 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:12:57.646945 23511 raft_consensus.cc:359] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.647665 23511 raft_consensus.cc:385] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.647943 23511 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.648571 23511 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.650995 23511 ts_tablet_manager.cc:1434] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.001s
I20260709 12:12:57.666204 23511 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:57.673761 23512 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:57.674512 23512 ts_tablet_manager.cc:1403] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.201s	user 0.018s	sys 0.003s
I20260709 12:12:57.692353 23890 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.692956 23890 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.699719 23890 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.706398 23469 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.708966 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.697873 23911 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.694653 23512 raft_consensus.cc:359] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.710423 23911 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.713351 23512 raft_consensus.cc:385] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.717545 23391 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:57.722664 23219 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:57.725364 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.729617 23910 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.739136 23219 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:57.739728 23910 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.740255 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:57.731775 23511 tablet_bootstrap.cc:654] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.736771 23911 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.737833 23512 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.742452 23467 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 0.
I20260709 12:12:57.748548 23918 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:12:57.749717 23319 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:57.755240 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:57.756521 23911 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:57.756888 23911 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.756263 23512 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.757265 23911 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.759049 23917 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:57.766501 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:57.769775 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:57.771787 23916 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:57.772289 23916 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.772698 23916 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.775920 23910 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.807017 23512 ts_tablet_manager.cc:1434] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.132s	user 0.009s	sys 0.000s
I20260709 12:12:57.808128 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:57.816251 23911 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.817466 23391 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:57.837802 23911 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:57.844339 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:57.851284 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:57.851945 23321 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:57.852838 23512 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:57.817519 23916 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.855749 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.875154 23468 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:57.876932 23916 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:57.896265 23321 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:12:57.897770 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.898741 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:57.899080 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:57.908917 23315 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:12:57.902618 23895 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:57.910815 23895 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:12:57.915969 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:57.911691 23895 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.920444 23887 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:57.921233 23512 tablet_bootstrap.cc:654] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.923374 23511 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:57.923805 23511 ts_tablet_manager.cc:1403] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.267s	user 0.017s	sys 0.031s
I20260709 12:12:57.925208 23219 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:12:57.941938 23920 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:12:57.949530 23511 raft_consensus.cc:359] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.950237 23511 raft_consensus.cc:385] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.950598 23511 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.951174 23511 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.951484 23887 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:12:57.960505 23512 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:57.960906 23512 ts_tablet_manager.cc:1403] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.108s	user 0.006s	sys 0.005s
I20260709 12:12:57.961948 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:57.963606 23918 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:57.965785 23511 ts_tablet_manager.cc:1434] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.042s	user 0.004s	sys 0.005s
I20260709 12:12:57.967760 23892 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:57.968735 23511 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:57.970504 23512 raft_consensus.cc:359] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.971179 23512 raft_consensus.cc:385] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:57.971475 23512 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:57.968987 23218 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:57.975521 23511 tablet_bootstrap.cc:654] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:57.981220 23512 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.984448 23512 ts_tablet_manager.cc:1434] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 12:12:57.992838 23895 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.993505 23895 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:57.996520 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:57.997787 23512 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
W20260709 12:12:57.997987 23360 leader_election.cc:343] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:57.997926 23895 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.000837 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:12:58.002470 23362 leader_election.cc:343] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.003002 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:58.004310 23895 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.004856 23512 tablet_bootstrap.cc:654] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.009690 23467 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:58.011369 23895 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.018685 23511 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.019354 23511 ts_tablet_manager.cc:1403] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.051s	user 0.021s	sys 0.005s
I20260709 12:12:58.023206 23511 raft_consensus.cc:359] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.024132 23511 raft_consensus.cc:385] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.024473 23511 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
W20260709 12:12:58.029729 23287 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:12:58.029927 23892 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.030397 23892 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.030654 23887 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.032739 23892 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.025300 23511 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.035862 23512 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:58.036298 23512 ts_tablet_manager.cc:1403] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.012s
I20260709 12:12:58.037333 23511 ts_tablet_manager.cc:1434] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.005s
I20260709 12:12:58.040313 23511 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.046212 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.048823 23387 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.058921 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:58.059578 23467 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.062521 23917 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.062973 23917 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.082079 23917 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.105405 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.106119 23395 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:58.107991 23512 raft_consensus.cc:359] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.108695 23512 raft_consensus.cc:385] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.108966 23512 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.109133 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:58.109649 23512 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.109995 23779 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.110730 23779 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.111105 23779 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.112042 23318 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:12:58.120383 23779 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:58.121217 23267 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:12:58.121732 23511 tablet_bootstrap.cc:654] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.116679 23287 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.129117 23512 ts_tablet_manager.cc:1434] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:12:58.130682 23512 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:58.128549 23892 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.128279 23779 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.124912 23893 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:12:58.146142 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.146937 23395 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.132164 23892 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.151860 23892 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.164438 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:58.165381 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:58.187973 23717 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.188513 23717 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.190634 23717 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.201874 23892 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.203225 23315 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:58.222260 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.223031 23387 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.224931 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.227715 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.232539 23787 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.232957 23787 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.233275 23787 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.232530 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:58.234608 23467 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:58.214952 23892 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.250396 23512 tablet_bootstrap.cc:654] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.252720 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:58.253908 23395 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:58.258026 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:12:58.259263 23916 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.259745 23916 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:58.260538 23916 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.263358 23287 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:58.264020 23887 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.265797 23787 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.268282 23511 tablet_bootstrap.cc:492] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.268388 23787 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.268730 23511 ts_tablet_manager.cc:1403] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.229s	user 0.024s	sys 0.031s
W20260709 12:12:58.269722 23287 leader_election.cc:343] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.274883 23511 raft_consensus.cc:359] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.275549 23511 raft_consensus.cc:385] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.275825 23511 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.276453 23511 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.278818 23511 ts_tablet_manager.cc:1434] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:12:58.279856 23511 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.284462 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.285123 23391 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.286036 23511 tablet_bootstrap.cc:654] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.294850 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:58.299022 23391 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:58.301072 23511 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.301760 23511 ts_tablet_manager.cc:1403] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 12:12:58.301070 23219 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:58.303912 23287 leader_election.cc:343] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.304562 23511 raft_consensus.cc:359] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.305272 23511 raft_consensus.cc:385] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.305739 23511 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.305994 23512 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:58.306757 23512 ts_tablet_manager.cc:1403] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.176s	user 0.021s	sys 0.016s
I20260709 12:12:58.306897 23511 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.309676 23511 ts_tablet_manager.cc:1434] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:12:58.310712 23511 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.316196 23916 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.316483 23511 tablet_bootstrap.cc:654] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.316694 23916 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.318815 23916 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.320920 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.321748 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:58.322501 23887 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.323009 23887 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:58.323856 23887 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.328202 23512 raft_consensus.cc:359] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.329245 23512 raft_consensus.cc:385] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.329634 23512 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.330619 23512 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.333067 23512 ts_tablet_manager.cc:1434] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:12:58.334599 23512 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:58.344348 23512 tablet_bootstrap.cc:654] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.345187 23511 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.345872 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:58.348021 23511 ts_tablet_manager.cc:1403] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.023s
I20260709 12:12:58.352348 23511 raft_consensus.cc:359] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.353227 23511 raft_consensus.cc:385] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.353586 23511 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.354553 23511 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.357913 23511 ts_tablet_manager.cc:1434] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:12:58.359830 23511 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.362426 23434 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:58.364068 23917 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.365370 23911 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:58.369971 23511 tablet_bootstrap.cc:654] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.372593 23512 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:58.373030 23512 ts_tablet_manager.cc:1403] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.010s
I20260709 12:12:58.380558 23512 raft_consensus.cc:359] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.381217 23512 raft_consensus.cc:385] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.381508 23512 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.382263 23512 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.383922 23511 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.383563 23217 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.384361 23511 ts_tablet_manager.cc:1403] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.000s
I20260709 12:12:58.384306 23512 ts_tablet_manager.cc:1434] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:12:58.385612 23512 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Bootstrap starting.
I20260709 12:12:58.387102 23511 raft_consensus.cc:359] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.388072 23511 raft_consensus.cc:385] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.388376 23511 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.389091 23511 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.391631 23512 tablet_bootstrap.cc:654] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.392742 23896 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.399155 23511 ts_tablet_manager.cc:1434] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:12:58.400153 23511 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.401945 23916 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.402434 23916 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.404881 23916 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.406960 23511 tablet_bootstrap.cc:654] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.406872 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:58.414203 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.416682 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:58.418105 23916 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.420619 23887 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.421288 23887 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.426681 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.427605 23395 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.429056 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.429754 23787 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.430140 23787 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.430460 23787 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.432578 23512 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: No bootstrap required, opened a new log
I20260709 12:12:58.433034 23512 ts_tablet_manager.cc:1403] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.018s
I20260709 12:12:58.435737 23512 raft_consensus.cc:359] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.436398 23512 raft_consensus.cc:385] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.436658 23512 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.437098 23787 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.437461 23512 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.438416 23887 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.439385 23787 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.439568 23512 ts_tablet_manager.cc:1434] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:12:58.440896 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.441444 23395 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.441804 23511 tablet_bootstrap.cc:492] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.442234 23511 ts_tablet_manager.cc:1403] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.016s
I20260709 12:12:58.443852 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:58.444702 23511 raft_consensus.cc:359] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.445339 23511 raft_consensus.cc:385] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.445624 23511 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
W20260709 12:12:58.445832 23287 leader_election.cc:343] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.446249 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:58.446314 23511 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:12:58.447804 23287 leader_election.cc:343] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:12:58.448504 23511 ts_tablet_manager.cc:1434] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:12:58.449375 23511 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.454380 23511 tablet_bootstrap.cc:654] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.460973 23395 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:58.463034 23468 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.463909 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:12:58.466548 23787 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.467173 23787 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:58.467965 23787 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.469584 23511 tablet_bootstrap.cc:492] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.470052 23511 ts_tablet_manager.cc:1403] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.003s
I20260709 12:12:58.482249 23511 raft_consensus.cc:359] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.482997 23511 raft_consensus.cc:385] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.491024 23511 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.491727 23511 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.493176 23511 ts_tablet_manager.cc:1434] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:12:58.494136 23511 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Bootstrap starting.
I20260709 12:12:58.500440 23511 tablet_bootstrap.cc:654] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Neither blocks nor log segments found. Creating new log.
I20260709 12:12:58.505645 23511 tablet_bootstrap.cc:492] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: No bootstrap required, opened a new log
I20260709 12:12:58.506108 23511 ts_tablet_manager.cc:1403] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.002s
I20260709 12:12:58.508719 23511 raft_consensus.cc:359] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.509943 23511 raft_consensus.cc:385] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:12:58.510212 23511 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Initialized, Role: FOLLOWER
I20260709 12:12:58.510854 23511 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.512666 23511 ts_tablet_manager.cc:1434] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 12:12:58.513213 23217 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.583683 23468 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.590956 23787 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.591974 23787 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.594450 23787 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.595389 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.596015 23391 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.596218 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:58.597697 23468 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.598011 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.598193 23916 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.598850 23717 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.598681 23916 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.599175 23717 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.599496 23717 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.600706 23916 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.602188 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.602754 23391 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:58.603094 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:58.603927 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:58.604225 23787 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.604795 23916 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.605139 23916 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.605949 23717 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.606524 23916 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.607069 23321 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:58.612558 23717 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.614949 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:58.617024 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.617637 23391 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.620328 23916 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.622905 23916 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.604803 23787 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.624716 23787 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.625589 23468 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 1: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:12:58.626693 23391 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:58.627674 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.628288 23395 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.628434 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:12:58.628957 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.629127 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.629912 23387 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:58.629944 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:58.630563 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.630750 23321 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:58.631281 23787 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.631335 23717 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.629050 23468 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 0.
I20260709 12:12:58.631851 23717 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.632302 23717 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.631843 23787 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:58.633626 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:58.633363 23787 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.634586 23916 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:12:58.639247 23717 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.641258 23717 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:12:58.644363 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.644937 23387 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.652480 23217 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.653357 23387 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:58.653990 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:12:58.654541 23468 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.654794 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:12:58.655529 23787 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.656004 23787 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:12:58.656805 23787 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.660821 23468 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:12:58.669428 23217 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:58.708590 23468 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.714330 23787 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.727109 23717 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:58.762451 23467 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.768702 23916 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.769235 23916 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.771311 23916 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.771872 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:58.772245 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:58.772545 23321 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:58.772859 23387 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 0.
I20260709 12:12:58.773694 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:58.774410 23916 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:12:58.774749 23916 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.775022 23916 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.783704 23917 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.781661 23916 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.789011 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:12:58.789544 23395 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.792569 23916 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:58.793825 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:12:58.794474 23315 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:58.807966 23916 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:58.817010 23315 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:58.819234 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:12:58.821362 23962 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:12:58.821892 23962 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:12:58.822772 23962 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:58.823727 23395 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:12:58.850849 23219 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 0 to 1, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:59.010843 23916 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.011492 23916 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:12:59.014931 23916 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:12:59.018007 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:12:59.018859 23390 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 1: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:12:59.022845 23467 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:12:59.023505 23717 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.024472 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:12:59.032773 23717 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:59.033421 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:12:59.050158 23916 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:12:59.052963 23787 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:12:59.061671 23717 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.088903 23717 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.115969 23899 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.125108 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:12:59.151573 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:59.154165 23971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:12:59.202883 23974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:12:59.204388 22107 server_base.cc:1061] running on GCE node
I20260709 12:12:59.205849 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:12:59.206120 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:12:59.206358 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599179206338 us; error 0 us; skew 500 ppm
I20260709 12:12:59.207132 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:12:59.215415 22107 webserver.cc:533] Webserver started at http://127.21.150.196:33957/ using document root <none> and password file <none>
I20260709 12:12:59.215973 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:12:59.216204 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:12:59.216532 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:12:59.217819 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-3-root/instance:
uuid: "54640ab8abeb4c949426d66c7e5aed87"
format_stamp: "Formatted at 2026-07-09 12:12:59 on dist-test-slave-zctp"
I20260709 12:12:59.222440 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:12:59.226691 23978 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:59.227767 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 12:12:59.228096 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
uuid: "54640ab8abeb4c949426d66c7e5aed87"
format_stamp: "Formatted at 2026-07-09 12:12:59 on dist-test-slave-zctp"
I20260709 12:12:59.228369 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:12:59.260676 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:12:59.262137 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:12:59.263959 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:12:59.266960 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:12:59.267241 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:59.267540 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:12:59.267745 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:12:59.358682 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.196:40405
I20260709 12:12:59.358776 24040 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.196:40405 every 8 connection(s)
I20260709 12:12:59.374038 23962 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.384974 22107 auto_leader_rebalancer-test.cc:439] add a tserver: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:12:59.398006 23962 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:12:59.408608 24041 heartbeater.cc:344] Connected to a master server at 127.21.150.254:44861
I20260709 12:12:59.409260 24041 heartbeater.cc:461] Registering TS with master...
I20260709 12:12:59.410610 24041 heartbeater.cc:507] Master 127.21.150.254:44861 requested a full tablet report, sending...
I20260709 12:12:59.414439 23219 ts_manager.cc:194] Registered new tserver with Master: 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:12:59.416756 23219 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36890
I20260709 12:13:00.293286 23467 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:00.302420 23390 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:00.304107 23962 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.305189 23315 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:00.306882 23962 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.312358 24046 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.314133 23390 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:00.318446 23436 consensus_peers.cc:597] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19b18a93b6e8478e9f44ce88b342ab54. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:00.325796 23321 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.334622 23219 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.364362 23320 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:00.399216 23468 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:00.405063 23390 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:00.409538 23717 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:00.411517 24062 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:00.420809 23287 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 29bc5916fff44ea3b1dcd31091d822b9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:00.421309 24041 heartbeater.cc:499] Master 127.21.150.254:44861 was elected leader, sending a full tablet report...
I20260709 12:13:00.424890 24062 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.425505 23468 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.433497 23395 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.445791 23217 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.478986 23395 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:00.486781 23320 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:00.488291 24048 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.490782 23467 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:00.496562 23896 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:00.536418 23362 consensus_peers.cc:597] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 446014759a9c4a0f9c99759951367083. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:00.541747 24071 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.543956 23468 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.546049 23320 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.559890 23395 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:00.564741 23218 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.581589 23467 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:00.581756 23362 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a658e02588314c9ca8a73f5eb423fb27. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:00.584220 23320 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:00.585078 23896 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:00.587716 24071 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:00.593822 23896 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.595872 23468 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.597661 23320 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.615386 23217 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:00.878160 24077 ts_tablet_manager.cc:933] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:00.880529 24077 tablet_copy_client.cc:323] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:00.906008 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 19b18a93b6e8478e9f44ce88b342ab54 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:00.906561 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 19b18a93b6e8478e9f44ce88b342ab54 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-19b18a93b6e8478e9f44ce88b342ab54
I20260709 12:13:00.916080 23480 tablet_copy_source_session.cc:215] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:00.921118 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19b18a93b6e8478e9f44ce88b342ab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:00.927788 24081 ts_tablet_manager.cc:933] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:00.939373 24077 tablet_copy_client.cc:806] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:00.940280 24077 tablet_copy_client.cc:670] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:00.942800 24081 tablet_copy_client.cc:323] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:00.946445 24083 ts_tablet_manager.cc:933] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:00.948307 24077 tablet_copy_client.cc:538] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:00.953126 24083 tablet_copy_client.cc:323] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:00.961537 24077 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:00.976266 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 446014759a9c4a0f9c99759951367083 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:00.976917 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 446014759a9c4a0f9c99759951367083 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-446014759a9c4a0f9c99759951367083
I20260709 12:13:00.990442 23406 tablet_copy_source_session.cc:215] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:00.994335 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446014759a9c4a0f9c99759951367083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:01.005038 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 29bc5916fff44ea3b1dcd31091d822b9 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:01.005723 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 29bc5916fff44ea3b1dcd31091d822b9 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-29bc5916fff44ea3b1dcd31091d822b9
I20260709 12:13:01.006393 24077 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:01.007490 24077 tablet_bootstrap.cc:492] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:01.008167 24077 ts_tablet_manager.cc:1403] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.008s
I20260709 12:13:01.011029 24077 raft_consensus.cc:359] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.011924 24077 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:01.012571 24077 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.013559 24081 tablet_copy_client.cc:806] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:01.014274 24081 tablet_copy_client.cc:670] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:01.017195 24077 ts_tablet_manager.cc:1434] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 12:13:01.018107 23331 tablet_copy_source_session.cc:215] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:01.020017 24081 tablet_copy_client.cc:538] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:01.021814 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-19b18a93b6e8478e9f44ce88b342ab54 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:01.022359 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-19b18a93b6e8478e9f44ce88b342ab54 on tablet 19b18a93b6e8478e9f44ce88b342ab54 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:01.022521 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bc5916fff44ea3b1dcd31091d822b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:01.029889 24081 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:01.039991 24083 tablet_copy_client.cc:806] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:01.040692 24083 tablet_copy_client.cc:670] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:01.055745 24083 tablet_copy_client.cc:538] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:01.071642 24083 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:01.072244 24081 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:01.076457 24081 tablet_bootstrap.cc:492] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:01.078166 24081 ts_tablet_manager.cc:1403] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.049s	user 0.024s	sys 0.004s
I20260709 12:13:01.081785 24081 raft_consensus.cc:359] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.086402 24081 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:01.087119 24081 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.092355 24081 ts_tablet_manager.cc:1434] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:13:01.094372 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-446014759a9c4a0f9c99759951367083 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:01.094820 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-446014759a9c4a0f9c99759951367083 on tablet 446014759a9c4a0f9c99759951367083 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:01.102910 24083 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:01.103866 24083 tablet_bootstrap.cc:492] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:01.104552 24083 ts_tablet_manager.cc:1403] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.000s
I20260709 12:13:01.109776 24083 raft_consensus.cc:359] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.110742 24083 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:01.111438 24083 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.115346 24083 ts_tablet_manager.cc:1434] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:13:01.117847 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-29bc5916fff44ea3b1dcd31091d822b9 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:01.118294 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-29bc5916fff44ea3b1dcd31091d822b9 on tablet 29bc5916fff44ea3b1dcd31091d822b9 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:01.144698 24083 ts_tablet_manager.cc:933] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:01.146308 24083 tablet_copy_client.cc:323] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:01.147873 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet a658e02588314c9ca8a73f5eb423fb27 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:01.148322 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet a658e02588314c9ca8a73f5eb423fb27 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-a658e02588314c9ca8a73f5eb423fb27
I20260709 12:13:01.155154 23406 tablet_copy_source_session.cc:215] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:01.157925 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a658e02588314c9ca8a73f5eb423fb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:01.170217 24083 tablet_copy_client.cc:806] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:01.171132 24083 tablet_copy_client.cc:670] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:01.178874 24083 tablet_copy_client.cc:538] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:01.190027 24083 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:01.210309 24083 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:01.211197 24083 tablet_bootstrap.cc:492] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:01.211921 24083 ts_tablet_manager.cc:1403] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.022s	user 0.021s	sys 0.000s
I20260709 12:13:01.214975 24083 raft_consensus.cc:359] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.215818 24083 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:01.216444 24083 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:01.220865 24083 ts_tablet_manager.cc:1434] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:13:01.222608 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-a658e02588314c9ca8a73f5eb423fb27 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:01.223037 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-a658e02588314c9ca8a73f5eb423fb27 on tablet a658e02588314c9ca8a73f5eb423fb27 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:01.272723 24016 raft_consensus.cc:1217] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:01.382809 24016 raft_consensus.cc:1217] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:01.431921 24016 raft_consensus.cc:1217] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:01.659642 24016 raft_consensus.cc:1217] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:01.787308 23962 raft_consensus.cc:1064] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:01.790462 23962 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:01.797974 24016 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.799430 23387 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.799702 24093 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.802073 24098 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.810415 23320 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.815323 24093 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:01.834731 24099 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:01.836498 24016 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:01.842231 23321 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:01.856462 23218 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:01.862087 23387 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:01.892578 24071 raft_consensus.cc:1064] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:01.897850 24071 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:01.909540 24016 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.911681 24069 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.913022 23468 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.917685 24069 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.939778 23320 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:01.944564 24096 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:01.992314 23467 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.002319 24016 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.005373 23395 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.009172 24093 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:02.011859 24093 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.022140 23962 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.026026 23468 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.022158 24096 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.033597 24016 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.051254 23395 raft_consensus.cc:2955] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.060680 23787 raft_consensus.cc:1064] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:02.064843 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19b18a93b6e8478e9f44ce88b342ab54 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:02.069768 23787 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.076085 23216 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.091141 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 19b18a93b6e8478e9f44ce88b342ab54 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:02.105799 24111 tablet_replica.cc:333] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:02.106856 24111 raft_consensus.cc:2243] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:02.126526 24016 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.162268 24111 raft_consensus.cc:2272] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:02.180622 24111 ts_tablet_manager.cc:1916] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:02.184365 24016 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:02.221153 24111 ts_tablet_manager.cc:1929] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:02.221796 24111 log.cc:1199] T 19b18a93b6e8478e9f44ce88b342ab54 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/19b18a93b6e8478e9f44ce88b342ab54
I20260709 12:13:02.122303 23320 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.251231 23391 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:02.252978 24062 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.224951 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 19b18a93b6e8478e9f44ce88b342ab54 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:13:02.292667 23204 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67882.8us
I20260709 12:13:02.301553 23219 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.307441 23468 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:02.309476 24066 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.337247 24118 raft_consensus.cc:1064] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:02.339787 24118 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.342561 24062 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.354151 23462 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.375342 23389 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.376412 23787 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.387261 23217 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.390817 24015 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.392633 24071 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.397456 24016 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.410872 23321 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.413904 24113 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.416411 23321 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:02.409124 23463 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:02.420375 24096 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:02.420558 23389 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.424153 24015 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:02.429608 24116 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.430253 24069 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:02.439888 23320 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.443463 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 446014759a9c4a0f9c99759951367083 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:02.447662 23321 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.453529 24016 raft_consensus.cc:2955] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.457662 24113 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:02.475075 24071 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.479212 23320 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.483212 24016 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:02.500797 23218 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.504235 23463 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:02.511544 24119 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:02.515084 24119 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:02.515414 23463 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.525779 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 446014759a9c4a0f9c99759951367083 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:02.527288 24114 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.529814 24125 tablet_replica.cc:333] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:02.531068 24125 raft_consensus.cc:2243] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:02.538157 24016 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.545006 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 29bc5916fff44ea3b1dcd31091d822b9 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:02.531744 24125 raft_consensus.cc:2272] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:02.552738 23218 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.566273 24125 ts_tablet_manager.cc:1916] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:02.571262 23463 raft_consensus.cc:2955] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.578096 23219 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.554345 24015 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.593333 24125 ts_tablet_manager.cc:1929] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:02.593842 24125 log.cc:1199] T 446014759a9c4a0f9c99759951367083 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/446014759a9c4a0f9c99759951367083
I20260709 12:13:02.595796 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 446014759a9c4a0f9c99759951367083 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:02.599542 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 29bc5916fff44ea3b1dcd31091d822b9 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:02.604715 23387 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:02.610247 24126 tablet_replica.cc:333] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:02.611284 24126 raft_consensus.cc:2243] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:02.612043 24126 raft_consensus.cc:2272] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:02.614800 24015 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.617885 24096 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.619175 24126 ts_tablet_manager.cc:1916] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:02.620400 23463 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:02.623518 24113 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:02.631771 24071 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.638006 24126 ts_tablet_manager.cc:1929] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:02.638533 24126 log.cc:1199] T 29bc5916fff44ea3b1dcd31091d822b9 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/29bc5916fff44ea3b1dcd31091d822b9
I20260709 12:13:02.638106 24016 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.645854 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 29bc5916fff44ea3b1dcd31091d822b9 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:02.650245 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a658e02588314c9ca8a73f5eb423fb27 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:02.653640 23217 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.657284 23463 raft_consensus.cc:2955] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:02.668473 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet a658e02588314c9ca8a73f5eb423fb27 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:02.671844 24111 tablet_replica.cc:333] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:02.672820 24111 raft_consensus.cc:2243] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:02.673482 24111 raft_consensus.cc:2272] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:02.676662 24111 ts_tablet_manager.cc:1916] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:02.691107 24111 ts_tablet_manager.cc:1929] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:02.691519 24111 log.cc:1199] T a658e02588314c9ca8a73f5eb423fb27 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/a658e02588314c9ca8a73f5eb423fb27
I20260709 12:13:02.692837 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet a658e02588314c9ca8a73f5eb423fb27 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:13:03.196355 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2040 after lost signal to thread 22110
W20260709 12:13:03.197405 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800070440 after lost signal to thread 23253
W20260709 12:13:03.198575 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800309800 after lost signal to thread 23345
W20260709 12:13:03.199947 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08002950a0 after lost signal to thread 23420
W20260709 12:13:03.200925 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08002a8740 after lost signal to thread 23494
W20260709 12:13:03.201982 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080024a600 after lost signal to thread 24040
I20260709 12:13:03.763777 23320 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:03.771464 23462 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:03.771850 23389 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:03.772179 23287 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 350816ae62fd4fb598a9ea6c84adf933. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:03.773066 24127 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.774001 24129 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.783568 24129 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:03.787097 23389 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:03.804277 23462 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:03.809100 23217 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:03.819723 23391 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:03.859315 23362 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbf1b9383e214ea7bb12a7f8784bffd1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:03.859969 23320 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:03.859361 23463 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:03.866050 24128 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.876354 24128 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:03.934784 24113 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:03.938946 23462 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:03.957230 23320 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:03.961071 23219 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:03.977890 23315 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.007611 23462 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:04.009274 24127 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:04.027334 23391 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:04.035872 24129 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:04.040234 23287 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9689b2f60f934eb6b1c067507423b816. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:04.052651 24129 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.055371 23468 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.069497 23391 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.076905 23389 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.083067 23218 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:04.162963 23362 consensus_peers.cc:597] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 19d9facb5378417aaa3279b3dd9f02a9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:04.163234 23462 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:04.165474 24118 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:13:04.231640 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 64588.7us
I20260709 12:13:04.233625 23315 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:04.237828 24141 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:04.317766 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66376us
I20260709 12:13:04.307737 23463 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.311702 24141 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:04.375903 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56583.4us
I20260709 12:13:04.385273 23218 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:04.413777 23315 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:04.462898 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43974us
W20260709 12:13:04.482206 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62383.7us
W20260709 12:13:04.511116 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 90135.3us
I20260709 12:13:04.526422 24162 ts_tablet_manager.cc:933] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:04.534869 24162 tablet_copy_client.cc:323] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:04.584434 24168 ts_tablet_manager.cc:933] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:04.589740 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 350816ae62fd4fb598a9ea6c84adf933 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:04.590476 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 350816ae62fd4fb598a9ea6c84adf933 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-350816ae62fd4fb598a9ea6c84adf933
I20260709 12:13:04.598237 23331 tablet_copy_source_session.cc:215] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:04.601478 24162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 350816ae62fd4fb598a9ea6c84adf933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:04.650218 24162 tablet_copy_client.cc:806] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:04.650914 24162 tablet_copy_client.cc:670] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:04.653810 24170 ts_tablet_manager.cc:933] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:04.655056 24162 tablet_copy_client.cc:538] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:04.655072 24168 tablet_copy_client.cc:323] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:04.661444 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet dbf1b9383e214ea7bb12a7f8784bffd1 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:04.662026 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet dbf1b9383e214ea7bb12a7f8784bffd1 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-dbf1b9383e214ea7bb12a7f8784bffd1
I20260709 12:13:04.666656 24162 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:04.670528 23406 tablet_copy_source_session.cc:215] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:04.672215 24170 tablet_copy_client.cc:323] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:04.674064 24168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf1b9383e214ea7bb12a7f8784bffd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:04.685523 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 9689b2f60f934eb6b1c067507423b816 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:04.686149 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 9689b2f60f934eb6b1c067507423b816 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-9689b2f60f934eb6b1c067507423b816
I20260709 12:13:04.696455 24168 tablet_copy_client.cc:806] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:04.701956 24168 tablet_copy_client.cc:670] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:04.709268 24168 tablet_copy_client.cc:538] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:04.713744 23331 tablet_copy_source_session.cc:215] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:04.717128 24170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9689b2f60f934eb6b1c067507423b816. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:04.738555 24168 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:04.751004 24162 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:04.755398 24162 tablet_bootstrap.cc:492] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:04.763230 24162 ts_tablet_manager.cc:1403] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.097s	user 0.025s	sys 0.000s
I20260709 12:13:04.754313 24170 tablet_copy_client.cc:806] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:04.764123 24170 tablet_copy_client.cc:670] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:04.768702 24170 tablet_copy_client.cc:538] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:04.778139 24162 raft_consensus.cc:359] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.788321 24162 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:04.790997 24162 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.790230 24170 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:04.807283 24173 ts_tablet_manager.cc:933] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:04.823006 24162 ts_tablet_manager.cc:1434] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:13:04.825284 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-350816ae62fd4fb598a9ea6c84adf933 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:04.825768 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-350816ae62fd4fb598a9ea6c84adf933 on tablet 350816ae62fd4fb598a9ea6c84adf933 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:04.846966 24168 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:04.848057 24168 tablet_bootstrap.cc:492] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:04.854633 24168 ts_tablet_manager.cc:1403] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.012s
I20260709 12:13:04.864475 24168 raft_consensus.cc:359] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.865368 24168 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:04.870034 24173 tablet_copy_client.cc:323] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:04.870260 24168 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.873672 24168 ts_tablet_manager.cc:1434] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 12:13:04.882664 23405 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-dbf1b9383e214ea7bb12a7f8784bffd1 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:04.883291 23405 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-dbf1b9383e214ea7bb12a7f8784bffd1 on tablet dbf1b9383e214ea7bb12a7f8784bffd1 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:04.873759 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 19d9facb5378417aaa3279b3dd9f02a9 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:04.885882 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 19d9facb5378417aaa3279b3dd9f02a9 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-19d9facb5378417aaa3279b3dd9f02a9
I20260709 12:13:04.909900 23406 tablet_copy_source_session.cc:215] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:04.913443 24173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19d9facb5378417aaa3279b3dd9f02a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:04.949493 24173 tablet_copy_client.cc:806] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:04.950132 24173 tablet_copy_client.cc:670] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:04.979094 24173 tablet_copy_client.cc:538] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:04.980520 24170 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:04.981549 24170 tablet_bootstrap.cc:492] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:04.982246 24170 ts_tablet_manager.cc:1403] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.192s	user 0.028s	sys 0.035s
I20260709 12:13:04.989552 24173 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:04.985351 24170 raft_consensus.cc:359] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:04.990283 24170 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:04.991688 24170 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:05.003386 24170 ts_tablet_manager.cc:1434] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.005s
I20260709 12:13:05.005487 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-9689b2f60f934eb6b1c067507423b816 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:05.005918 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-9689b2f60f934eb6b1c067507423b816 on tablet 9689b2f60f934eb6b1c067507423b816 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:05.012197 24016 raft_consensus.cc:1217] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:05.025391 24173 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:05.026327 24173 tablet_bootstrap.cc:492] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:05.026985 24173 ts_tablet_manager.cc:1403] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.006s
I20260709 12:13:05.030133 24173 raft_consensus.cc:359] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:05.030854 24173 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:05.074954 24173 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:05.083309 24173 ts_tablet_manager.cc:1434] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.008s
I20260709 12:13:05.088366 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-19d9facb5378417aaa3279b3dd9f02a9 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:05.089277 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-19d9facb5378417aaa3279b3dd9f02a9 on tablet 19d9facb5378417aaa3279b3dd9f02a9 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:05.152562 24015 raft_consensus.cc:1217] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:05.228091 24015 raft_consensus.cc:1217] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:05.245527 24015 raft_consensus.cc:1217] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:05.599431 24141 raft_consensus.cc:1064] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:05.615769 24141 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:05.624131 24062 raft_consensus.cc:1064] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:05.626958 24062 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:05.671265 24015 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:05.677955 24181 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:05.679695 23469 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:05.695899 23321 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:05.698763 24166 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:05.714695 24166 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:05.738634 23469 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:05.738893 24016 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:05.739173 23388 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:13:05.822973 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 82030.5us
I20260709 12:13:05.823055 24152 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:05.826413 24152 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:05.828387 24129 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:05.855026 24166 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:05.864655 24118 raft_consensus.cc:1064] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:05.868788 24118 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:05.877544 24161 raft_consensus.cc:1064] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:05.881302 24161 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:05.915197 23321 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.010834 23463 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:06.009603 24016 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.044624 24015 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:06.094327 23321 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:13:06.098888 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 101411us
I20260709 12:13:06.102095 24197 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:13:06.109419 23219 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.112996 24193 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
W20260709 12:13:06.120034 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89295.6us
W20260709 12:13:06.128489 24187 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 12:13:06.136891 23469 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:06.152959 24014 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:06.155246 24198 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.159616 23387 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:06.158838 23468 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:06.177281 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84428.9us
I20260709 12:13:06.188395 24202 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:06.187891 24141 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.194027 23395 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.198875 24016 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.212510 24015 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.270512 23466 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.282850 24196 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:06.291679 23463 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.305313 24062 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:06.309814 24163 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:06.313694 23315 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.369210 23395 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:06.377331 24015 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.381131 24181 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:06.385228 24152 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.384810 23216 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.393369 23321 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.398769 24141 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.398347 23392 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.396217 24016 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.405258 24205 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.410235 23466 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.415529 24015 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.391868 23216 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.427721 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dbf1b9383e214ea7bb12a7f8784bffd1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:06.428673 23321 raft_consensus.cc:2955] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.467149 23218 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.473670 23218 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.490811 23321 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:06.480974 23392 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:06.497170 24015 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.500622 24015 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.500828 23466 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.502573 24152 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.503345 23466 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.503897 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet dbf1b9383e214ea7bb12a7f8784bffd1 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:06.504415 24205 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.505129 24062 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.505927 24141 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.510246 23321 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:06.513518 24141 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.516173 23466 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.525271 24016 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.525574 23468 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:06.529795 24016 raft_consensus.cc:2955] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.537906 24062 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:06.542542 24062 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.545907 24127 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:06.544040 24220 tablet_replica.cc:333] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:06.547959 24220 raft_consensus.cc:2243] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:06.548695 24220 raft_consensus.cc:2272] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:06.553632 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9689b2f60f934eb6b1c067507423b816 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:06.555634 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 19d9facb5378417aaa3279b3dd9f02a9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:06.564623 23218 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.566684 24220 ts_tablet_manager.cc:1916] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:06.569602 23219 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:06.585510 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 9689b2f60f934eb6b1c067507423b816 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:06.592295 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 19d9facb5378417aaa3279b3dd9f02a9 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:06.593896 24221 tablet_replica.cc:333] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:06.597747 24221 raft_consensus.cc:2243] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:06.598904 24221 raft_consensus.cc:2272] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:06.603917 24014 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.608489 24221 ts_tablet_manager.cc:1916] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:06.611984 24220 ts_tablet_manager.cc:1929] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:06.612527 24220 log.cc:1199] T dbf1b9383e214ea7bb12a7f8784bffd1 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/dbf1b9383e214ea7bb12a7f8784bffd1
I20260709 12:13:06.619621 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet dbf1b9383e214ea7bb12a7f8784bffd1 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:06.621261 24222 tablet_replica.cc:333] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:06.622469 24222 raft_consensus.cc:2243] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:06.623170 24222 raft_consensus.cc:2272] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:06.626703 23466 raft_consensus.cc:2955] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.630733 24222 ts_tablet_manager.cc:1916] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:06.644368 24221 ts_tablet_manager.cc:1929] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:06.644948 24221 log.cc:1199] T 9689b2f60f934eb6b1c067507423b816 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/9689b2f60f934eb6b1c067507423b816
I20260709 12:13:06.653872 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 9689b2f60f934eb6b1c067507423b816 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:06.660256 24198 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:06.749248 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89217.6us
I20260709 12:13:06.758267 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 350816ae62fd4fb598a9ea6c84adf933 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:06.768383 23462 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.769965 23219 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:06.774957 24222 ts_tablet_manager.cc:1929] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:06.775499 24222 log.cc:1199] T 19d9facb5378417aaa3279b3dd9f02a9 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/19d9facb5378417aaa3279b3dd9f02a9
I20260709 12:13:06.780972 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 19d9facb5378417aaa3279b3dd9f02a9 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:06.784128 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 350816ae62fd4fb598a9ea6c84adf933 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:06.785570 24221 tablet_replica.cc:333] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:06.786610 24221 raft_consensus.cc:2243] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:06.787389 24221 raft_consensus.cc:2272] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:06.787985 24014 raft_consensus.cc:2955] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:06.803319 24221 ts_tablet_manager.cc:1916] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:06.840279 24221 ts_tablet_manager.cc:1929] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:06.840799 24221 log.cc:1199] T 350816ae62fd4fb598a9ea6c84adf933 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/350816ae62fd4fb598a9ea6c84adf933
I20260709 12:13:06.842585 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 350816ae62fd4fb598a9ea6c84adf933 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:13:07.691637 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 194185us
I20260709 12:13:07.903342 23392 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:07.914474 23462 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:07.914682 23362 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c7ea21ce0804c45ac69bf6430bcfb45. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:07.916431 23321 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:07.916433 24225 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:07.919755 24205 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:07.931357 24141 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:07.933583 23462 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:07.941514 23321 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:07.969933 23462 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:07.969141 23218 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:07.994436 23315 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:07.997598 23436 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 32256a541626459187bbe0f5f7e6643c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:08.000767 24229 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.002794 23392 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:08.009706 24229 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:08.049757 24223 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.051589 23315 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.074482 23392 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.076912 23462 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.101022 23393 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:08.105199 23315 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:08.107779 24223 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:08.111091 23436 consensus_peers.cc:597] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd134c48fc14e60af68e7bff27733c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:08.111675 24223 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:08.136173 24146 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.151542 23321 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.156693 23315 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.158947 23392 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.186372 23219 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.243074 23219 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.252466 23463 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:08.255338 23393 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:08.256362 24245 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:08.258931 23287 consensus_peers.cc:597] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec46374214e6444fa4c5afabe5bb309a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:08.260317 24245 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:08.273633 24244 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.280233 23463 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.309173 23219 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:08.314674 23393 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:08.424412 24248 ts_tablet_manager.cc:933] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:08.427253 24248 tablet_copy_client.cc:323] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:08.429625 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 32256a541626459187bbe0f5f7e6643c from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:08.430218 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 32256a541626459187bbe0f5f7e6643c from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-32256a541626459187bbe0f5f7e6643c
I20260709 12:13:08.454605 23480 tablet_copy_source_session.cc:215] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:08.461695 24250 ts_tablet_manager.cc:933] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:08.467288 24248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32256a541626459187bbe0f5f7e6643c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:08.499694 24252 ts_tablet_manager.cc:933] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:08.512322 24250 tablet_copy_client.cc:323] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:08.514561 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:08.515081 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 7c7ea21ce0804c45ac69bf6430bcfb45 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-7c7ea21ce0804c45ac69bf6430bcfb45
I20260709 12:13:08.526736 24252 tablet_copy_client.cc:323] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:08.533272 24248 tablet_copy_client.cc:806] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:08.533352 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet abd134c48fc14e60af68e7bff27733c0 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:08.534160 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet abd134c48fc14e60af68e7bff27733c0 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-abd134c48fc14e60af68e7bff27733c0
I20260709 12:13:08.534257 24248 tablet_copy_client.cc:670] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:08.539004 23406 tablet_copy_source_session.cc:215] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:08.539495 24248 tablet_copy_client.cc:538] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:08.551491 24250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c7ea21ce0804c45ac69bf6430bcfb45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:08.563882 23480 tablet_copy_source_session.cc:215] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:08.567226 24252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd134c48fc14e60af68e7bff27733c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:08.568313 24248 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:08.585311 24250 tablet_copy_client.cc:806] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:08.586355 24252 tablet_copy_client.cc:806] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:08.586069 24250 tablet_copy_client.cc:670] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:08.587015 24252 tablet_copy_client.cc:670] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:08.590808 24250 tablet_copy_client.cc:538] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:08.595329 24252 tablet_copy_client.cc:538] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:08.610883 24248 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:08.611971 24248 tablet_bootstrap.cc:492] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:08.612778 24250 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:08.613761 24248 ts_tablet_manager.cc:1403] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.017s
W20260709 12:13:08.614825 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080018c380 after lost signal to thread 22110
W20260709 12:13:08.615923 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5580 after lost signal to thread 23253
W20260709 12:13:08.617131 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002a3f80 after lost signal to thread 23345
I20260709 12:13:08.617625 24248 raft_consensus.cc:359] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:08.618208 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080019cbc0 after lost signal to thread 23420
I20260709 12:13:08.618425 24248 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
W20260709 12:13:08.619300 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003148e0 after lost signal to thread 23494
I20260709 12:13:08.619072 24248 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:08.620465 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002d8820 after lost signal to thread 24040
I20260709 12:13:08.629382 24248 ts_tablet_manager.cc:1434] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.001s
I20260709 12:13:08.634454 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-32256a541626459187bbe0f5f7e6643c received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:08.634922 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-32256a541626459187bbe0f5f7e6643c on tablet 32256a541626459187bbe0f5f7e6643c with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:08.644542 24256 ts_tablet_manager.cc:933] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:08.614360 24252 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:08.660701 24256 tablet_copy_client.cc:323] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:08.662936 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet ec46374214e6444fa4c5afabe5bb309a from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:08.663488 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet ec46374214e6444fa4c5afabe5bb309a from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-ec46374214e6444fa4c5afabe5bb309a
I20260709 12:13:08.681547 23331 tablet_copy_source_session.cc:215] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:08.685204 24256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec46374214e6444fa4c5afabe5bb309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:08.697625 24252 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:08.698602 24252 tablet_bootstrap.cc:492] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:08.700345 24252 ts_tablet_manager.cc:1403] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.086s	user 0.019s	sys 0.004s
I20260709 12:13:08.704099 24252 raft_consensus.cc:359] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.705667 24252 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:08.706739 24252 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.710783 24252 ts_tablet_manager.cc:1434] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.004s
I20260709 12:13:08.715394 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-abd134c48fc14e60af68e7bff27733c0 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:08.715811 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-abd134c48fc14e60af68e7bff27733c0 on tablet abd134c48fc14e60af68e7bff27733c0 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:08.791961 24250 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:08.793037 24250 tablet_bootstrap.cc:492] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:08.793849 24250 ts_tablet_manager.cc:1403] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.181s	user 0.046s	sys 0.023s
I20260709 12:13:08.797217 24250 raft_consensus.cc:359] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.798260 24250 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:08.799139 24250 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.804215 24250 ts_tablet_manager.cc:1434] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:13:08.798026 24256 tablet_copy_client.cc:806] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:08.807088 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-7c7ea21ce0804c45ac69bf6430bcfb45 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:08.807575 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-7c7ea21ce0804c45ac69bf6430bcfb45 on tablet 7c7ea21ce0804c45ac69bf6430bcfb45 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:08.806710 24256 tablet_copy_client.cc:670] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:08.819366 24256 tablet_copy_client.cc:538] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:08.835093 24256 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:08.886169 24256 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:08.887156 24256 tablet_bootstrap.cc:492] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:08.887897 24256 ts_tablet_manager.cc:1403] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.019s
I20260709 12:13:08.891220 24256 raft_consensus.cc:359] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.892131 24256 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:08.892815 24256 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:08.900655 24256 ts_tablet_manager.cc:1434] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:13:08.902756 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ec46374214e6444fa4c5afabe5bb309a received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:08.903208 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ec46374214e6444fa4c5afabe5bb309a on tablet ec46374214e6444fa4c5afabe5bb309a with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:08.942312 24014 raft_consensus.cc:1217] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:09.002665 24014 raft_consensus.cc:1217] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:09.023063 24014 raft_consensus.cc:1217] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:13:09.257278 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43588.3us
I20260709 12:13:09.300679 24014 raft_consensus.cc:1217] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:09.496749 24261 raft_consensus.cc:1064] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:09.499751 24261 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:09.537622 24141 raft_consensus.cc:1064] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:09.539988 24141 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:09.571447 23466 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.573081 24205 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.577390 24014 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.580255 24205 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.593576 23319 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.603124 24141 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:13:09.669305 24198 raft_consensus.cc:1064] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:13:09.669881 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66982.2us
I20260709 12:13:09.671619 24198 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:09.719650 24277 raft_consensus.cc:1064] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:13:09.727988 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71635.3us
W20260709 12:13:09.747771 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72120.7us
I20260709 12:13:09.792189 23468 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:09.811779 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 97486.7us
I20260709 12:13:09.815357 23392 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.823352 24263 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.826159 24014 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.828781 24263 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.832916 23315 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:09.839741 23392 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.841252 23466 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.854218 24016 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.776459 24284 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:09.887753 24062 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:09.906240 23219 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:09.910785 23315 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.908573 24277 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:09.958899 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77345.1us
W20260709 12:13:09.975165 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 101761us
I20260709 12:13:09.985647 24261 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:09.948861 24287 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:13:09.996109 24014 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:09.997611 24280 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:13:10.014326 23390 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:10.015368 24270 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.016572 24016 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.029794 23392 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.034409 24282 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:10.035542 24289 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:10.051409 24015 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.097839 23319 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:10.244347 24289 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.248631 23392 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.270534 24223 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:10.270197 24288 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.276118 24014 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.355630 24014 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.386720 23391 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.397387 23319 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.397964 23217 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.415608 23392 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:10.434362 23319 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.434968 24014 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.441176 23466 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.451107 24292 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:10.454412 24292 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:10.457647 23218 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.457971 23216 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.458807 24205 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.465919 24014 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.471956 23315 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:10.474232 23319 raft_consensus.cc:2955] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.478219 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:10.492470 23466 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:10.496345 23463 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:10.505048 24016 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.507555 24289 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:10.509959 23391 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.512365 24277 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:10.519438 23321 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.526292 23219 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.529626 24016 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.528924 24223 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.532672 23391 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.534759 24289 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:10.537632 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c7ea21ce0804c45ac69bf6430bcfb45 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:10.538815 24016 raft_consensus.cc:2955] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.552570 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet abd134c48fc14e60af68e7bff27733c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:10.566286 24280 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.574231 23393 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.579056 23218 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:10.563413 24312 tablet_replica.cc:333] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:10.587767 24312 raft_consensus.cc:2243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:10.590260 24312 raft_consensus.cc:2272] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:10.590997 24014 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.593348 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet abd134c48fc14e60af68e7bff27733c0 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:10.606600 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 32256a541626459187bbe0f5f7e6643c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:10.607877 24016 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.614946 24312 ts_tablet_manager.cc:1916] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:10.630486 24313 tablet_replica.cc:333] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:10.638974 23219 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.655941 24198 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:10.661298 23321 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:10.667814 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 32256a541626459187bbe0f5f7e6643c with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:10.676339 24198 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:10.687384 24313 raft_consensus.cc:2243] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:10.687870 24314 tablet_replica.cc:333] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:10.688257 24313 raft_consensus.cc:2272] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:10.689303 24314 raft_consensus.cc:2243] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:10.692284 24313 ts_tablet_manager.cc:1916] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:10.697968 24314 raft_consensus.cc:2272] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:10.701227 24314 ts_tablet_manager.cc:1916] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:10.726456 24313 ts_tablet_manager.cc:1929] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:10.727057 24313 log.cc:1199] T abd134c48fc14e60af68e7bff27733c0 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/abd134c48fc14e60af68e7bff27733c0
I20260709 12:13:10.739035 24314 ts_tablet_manager.cc:1929] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:10.739573 24314 log.cc:1199] T 32256a541626459187bbe0f5f7e6643c P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/32256a541626459187bbe0f5f7e6643c
I20260709 12:13:10.764922 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet abd134c48fc14e60af68e7bff27733c0 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:10.775583 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 32256a541626459187bbe0f5f7e6643c (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:10.777936 24289 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:13:10.814543 24312 ts_tablet_manager.cc:1929] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:10.815066 24312 log.cc:1199] T 7c7ea21ce0804c45ac69bf6430bcfb45 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/7c7ea21ce0804c45ac69bf6430bcfb45
I20260709 12:13:10.817384 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 7c7ea21ce0804c45ac69bf6430bcfb45 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:10.819867 23319 raft_consensus.cc:2955] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.831295 24282 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.865139 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec46374214e6444fa4c5afabe5bb309a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:10.882030 23391 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.882033 24016 raft_consensus.cc:2955] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.916565 23219 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:10.934481 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet ec46374214e6444fa4c5afabe5bb309a with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:10.935634 24312 tablet_replica.cc:333] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:10.990445 24312 raft_consensus.cc:2243] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:10.991294 24312 raft_consensus.cc:2272] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:10.994971 24312 ts_tablet_manager.cc:1916] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:11.011618 24312 ts_tablet_manager.cc:1929] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:11.013114 24312 log.cc:1199] T ec46374214e6444fa4c5afabe5bb309a P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/ec46374214e6444fa4c5afabe5bb309a
I20260709 12:13:11.014874 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet ec46374214e6444fa4c5afabe5bb309a (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:12.058454 23463 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.072077 23392 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:13:12.074764 23436 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4de7d7863704646b55025720cc0da9c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:12.075474 24280 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.083446 23319 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:12.087608 24280 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.111269 24280 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.114872 23392 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.134474 23391 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.143644 23217 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:12.169243 23319 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:12.180815 23362 consensus_peers.cc:597] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb9a5906264b4cae9c8e888ad57c13c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:12.186208 23468 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:12.188930 24315 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.228077 23319 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:12.230252 24315 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:12.258387 24205 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.268476 23319 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.269132 23468 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.284653 23463 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.332376 23219 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:12.337486 23436 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea456647290e4959b61a6fa23ecfbfd4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:12.342897 23392 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:12.344552 24223 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.357321 23319 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:12.360000 24320 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:12.386953 24280 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.388625 23392 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.412416 23319 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.428673 23218 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.450615 23392 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:12.472666 23362 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: de7238eb389343cd954cb756bed5925f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:12.473610 23463 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:12.475109 24286 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:12.475416 23319 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:12.477196 24315 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:12.493412 24315 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.506507 23319 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.529536 23463 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:12.541452 23219 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:12.639573 24335 ts_tablet_manager.cc:933] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:12.657501 24335 tablet_copy_client.cc:323] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:12.659294 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet c4de7d7863704646b55025720cc0da9c from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:12.659825 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet c4de7d7863704646b55025720cc0da9c from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-c4de7d7863704646b55025720cc0da9c
I20260709 12:13:12.671977 23480 tablet_copy_source_session.cc:215] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:12.675830 24335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4de7d7863704646b55025720cc0da9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:12.692152 24335 tablet_copy_client.cc:806] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:12.692799 24335 tablet_copy_client.cc:670] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:12.697680 24335 tablet_copy_client.cc:538] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:12.712556 24335 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:12.770511 24338 ts_tablet_manager.cc:933] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:12.773058 24337 ts_tablet_manager.cc:933] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:12.783195 24337 tablet_copy_client.cc:323] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:12.785564 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet fb9a5906264b4cae9c8e888ad57c13c8 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:12.786243 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet fb9a5906264b4cae9c8e888ad57c13c8 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-fb9a5906264b4cae9c8e888ad57c13c8
I20260709 12:13:12.790582 24338 tablet_copy_client.cc:323] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:12.793198 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet ea456647290e4959b61a6fa23ecfbfd4 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:12.793825 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet ea456647290e4959b61a6fa23ecfbfd4 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-ea456647290e4959b61a6fa23ecfbfd4
I20260709 12:13:12.797581 24335 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:12.804700 23406 tablet_copy_source_session.cc:215] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:12.805465 23480 tablet_copy_source_session.cc:215] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:12.806125 24335 tablet_bootstrap.cc:492] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:12.806957 24335 ts_tablet_manager.cc:1403] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.095s	user 0.017s	sys 0.007s
I20260709 12:13:12.809007 24337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb9a5906264b4cae9c8e888ad57c13c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:12.810439 24338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea456647290e4959b61a6fa23ecfbfd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:12.810091 24335 raft_consensus.cc:359] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.811028 24335 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:12.811764 24335 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.823455 24335 ts_tablet_manager.cc:1434] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.008s
I20260709 12:13:12.829332 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c4de7d7863704646b55025720cc0da9c received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:12.829876 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c4de7d7863704646b55025720cc0da9c on tablet c4de7d7863704646b55025720cc0da9c with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:12.842319 24338 tablet_copy_client.cc:806] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:12.843052 24338 tablet_copy_client.cc:670] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:12.846844 24338 tablet_copy_client.cc:538] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:12.855978 24338 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:12.861711 24337 tablet_copy_client.cc:806] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:12.862522 24337 tablet_copy_client.cc:670] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:12.878895 24337 tablet_copy_client.cc:538] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:12.896487 24337 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:12.943490 24337 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:12.944650 24337 tablet_bootstrap.cc:492] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:12.945466 24337 ts_tablet_manager.cc:1403] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.012s
I20260709 12:13:12.949957 24337 raft_consensus.cc:359] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.950953 24337 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:12.952106 24337 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.955068 24337 ts_tablet_manager.cc:1434] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:13:12.957489 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-fb9a5906264b4cae9c8e888ad57c13c8 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:12.957937 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-fb9a5906264b4cae9c8e888ad57c13c8 on tablet fb9a5906264b4cae9c8e888ad57c13c8 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:12.954036 24338 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:12.962881 24338 tablet_bootstrap.cc:492] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:12.963496 24338 ts_tablet_manager.cc:1403] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.108s	user 0.022s	sys 0.010s
I20260709 12:13:12.967190 24338 raft_consensus.cc:359] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.968446 24338 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:12.973728 24338 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:12.976533 24338 ts_tablet_manager.cc:1434] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 12:13:12.978339 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ea456647290e4959b61a6fa23ecfbfd4 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:12.978772 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ea456647290e4959b61a6fa23ecfbfd4 on tablet ea456647290e4959b61a6fa23ecfbfd4 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:13.026001 24338 ts_tablet_manager.cc:933] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:13.046476 24338 tablet_copy_client.cc:323] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:13.048827 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet de7238eb389343cd954cb756bed5925f from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:13.049475 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet de7238eb389343cd954cb756bed5925f from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-de7238eb389343cd954cb756bed5925f
I20260709 12:13:13.057377 23406 tablet_copy_source_session.cc:215] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:13.061277 24338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7238eb389343cd954cb756bed5925f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:13.078495 24338 tablet_copy_client.cc:806] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:13.079110 24338 tablet_copy_client.cc:670] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:13.082917 24338 tablet_copy_client.cc:538] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:13.092455 24338 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:13.121412 24338 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:13.122375 24338 tablet_bootstrap.cc:492] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:13.123153 24338 ts_tablet_manager.cc:1403] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.031s	user 0.024s	sys 0.004s
I20260709 12:13:13.126147 24338 raft_consensus.cc:359] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:13.127103 24338 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:13.127825 24338 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:13.138487 24338 ts_tablet_manager.cc:1434] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.005s
I20260709 12:13:13.140837 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-de7238eb389343cd954cb756bed5925f received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:13.141369 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-de7238eb389343cd954cb756bed5925f on tablet de7238eb389343cd954cb756bed5925f with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:13.175485 24016 raft_consensus.cc:1217] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:13:13.354590 24016 raft_consensus.cc:1217] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:13.356426 24014 raft_consensus.cc:1217] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:13.459363 24016 raft_consensus.cc:1217] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:13.696697 24320 raft_consensus.cc:1064] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:13.699807 24320 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:13.721551 24016 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:13.723145 23392 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:13.723749 24223 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.725134 24280 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.754457 23319 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:13.778154 24014 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.773522 24223 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.801630 24280 raft_consensus.cc:1064] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:13.804821 24280 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:13.820152 24320 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.821951 23218 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.835211 23393 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.835445 24014 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.839273 23393 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.838707 24320 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:13.841748 24352 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.861549 23320 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.881367 23319 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.893265 24320 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:13.908952 24280 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.915357 24015 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.923971 24205 raft_consensus.cc:1064] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:13.926460 24205 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:13.932997 23392 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.955850 23320 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:13.958957 24015 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.960287 23468 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.961858 24315 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.964650 23321 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:13.966589 24334 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.967393 24205 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:13.974131 23468 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:13.975487 23218 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.011082 24015 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:14.012530 24280 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.013230 23392 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:14.014925 24354 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.022428 23468 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:14.026461 24334 raft_consensus.cc:1064] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:14.030115 24354 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.031384 24334 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:14.037382 24315 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.045109 23391 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.052806 24015 raft_consensus.cc:2955] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.045972 24014 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.078052 23462 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.081736 23320 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.084209 24016 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:14.086247 24205 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.090080 23466 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:14.092267 24334 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:14.101842 23321 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:14.123974 23219 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.131083 24205 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:14.132110 24016 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:14.133813 24280 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.148339 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c4de7d7863704646b55025720cc0da9c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:14.155934 24280 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.159680 24016 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.178318 23320 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:14.179555 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ea456647290e4959b61a6fa23ecfbfd4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:14.183779 24368 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.185613 23466 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.192797 23319 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.216253 24016 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.219058 23216 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.220724 23216 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.252324 23217 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.253463 24280 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.266197 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet ea456647290e4959b61a6fa23ecfbfd4 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:14.267437 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet c4de7d7863704646b55025720cc0da9c with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:14.265354 23321 raft_consensus.cc:2955] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.317039 24376 tablet_replica.cc:333] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:14.318389 24376 raft_consensus.cc:2243] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:14.318652 24377 tablet_replica.cc:333] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:14.319391 24376 raft_consensus.cc:2272] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:14.319880 24377 raft_consensus.cc:2243] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:14.321584 24377 raft_consensus.cc:2272] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:14.323721 24376 ts_tablet_manager.cc:1916] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:14.324731 23391 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:14.327611 24377 ts_tablet_manager.cc:1916] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:14.328099 23392 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:14.334247 24016 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:14.335440 23466 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:14.336418 24371 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:14.338815 24205 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:14.342770 24016 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:14.344336 24371 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:14.347466 24371 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.352146 24368 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.354278 24016 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.360474 24015 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.365177 23463 raft_consensus.cc:2955] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.370657 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet de7238eb389343cd954cb756bed5925f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:14.376008 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb9a5906264b4cae9c8e888ad57c13c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:14.380918 24376 ts_tablet_manager.cc:1929] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:14.381515 24376 log.cc:1199] T ea456647290e4959b61a6fa23ecfbfd4 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/ea456647290e4959b61a6fa23ecfbfd4
I20260709 12:13:14.383160 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet ea456647290e4959b61a6fa23ecfbfd4 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:14.382061 23216 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.386264 23218 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:14.401338 24377 ts_tablet_manager.cc:1929] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:13:14.417695 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet fb9a5906264b4cae9c8e888ad57c13c8 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:14.417330 23321 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:14.438772 24368 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:14.441530 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 107363us
I20260709 12:13:14.444515 23321 raft_consensus.cc:2955] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:14.441825 24378 tablet_replica.cc:333] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:14.449052 24378 raft_consensus.cc:2243] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:14.417359 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet de7238eb389343cd954cb756bed5925f with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:14.450057 24378 raft_consensus.cc:2272] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:14.455343 24378 ts_tablet_manager.cc:1916] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:14.462352 24377 log.cc:1199] T c4de7d7863704646b55025720cc0da9c P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/c4de7d7863704646b55025720cc0da9c
I20260709 12:13:14.465029 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet c4de7d7863704646b55025720cc0da9c (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:14.465586 24377 tablet_replica.cc:333] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:14.468528 24377 raft_consensus.cc:2243] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:14.473021 24377 raft_consensus.cc:2272] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:14.484508 24378 ts_tablet_manager.cc:1929] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:14.485146 24378 log.cc:1199] T fb9a5906264b4cae9c8e888ad57c13c8 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/fb9a5906264b4cae9c8e888ad57c13c8
I20260709 12:13:14.487111 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet fb9a5906264b4cae9c8e888ad57c13c8 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:14.490552 24377 ts_tablet_manager.cc:1916] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:14.513178 24377 ts_tablet_manager.cc:1929] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:14.513787 24377 log.cc:1199] T de7238eb389343cd954cb756bed5925f P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/de7238eb389343cd954cb756bed5925f
I20260709 12:13:14.518193 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet de7238eb389343cd954cb756bed5925f (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:15.553925 23315 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:15.570716 23468 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.572875 23391 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.572854 24375 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:15.574282 23287 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d736b5901fbd4de09b83cc09a8d00fd1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.578855 24375 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:15.590745 24375 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.604245 23466 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.617803 23391 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.635716 23321 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:15.629156 23218 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.656502 23392 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.659507 23466 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:15.666338 23287 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: baab486079784de4933d5f96a0a5321b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.668191 24375 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.670861 24347 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.684935 24382 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.686836 23466 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.697672 23391 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.724050 23466 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:15.734314 23321 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.735890 24354 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.737232 23391 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.751089 24381 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:13:15.745203 23436 consensus_peers.cc:597] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57b150e0c7494c95beb04642f466f450. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.755122 23219 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.766100 24354 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.777988 23391 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.791106 23468 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:15.794150 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:15.798108 23321 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.823879 23316 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:15.825299 23436 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1399c0b792a048cda2bcae40b23432c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:15.826407 23392 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:15.828222 24379 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.830504 24396 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:15.860496 24354 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.884472 23391 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.906719 23320 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:15.931887 23216 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:16.074697 24400 ts_tablet_manager.cc:933] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:16.086949 24400 tablet_copy_client.cc:323] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:16.089407 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet d736b5901fbd4de09b83cc09a8d00fd1 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:16.090107 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet d736b5901fbd4de09b83cc09a8d00fd1 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-d736b5901fbd4de09b83cc09a8d00fd1
I20260709 12:13:16.108911 23331 tablet_copy_source_session.cc:215] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.112701 24400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d736b5901fbd4de09b83cc09a8d00fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.150417 24400 tablet_copy_client.cc:806] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.151728 24400 tablet_copy_client.cc:670] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.167459 24400 tablet_copy_client.cc:538] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.170686 24402 ts_tablet_manager.cc:933] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:16.201943 24400 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:16.206440 24402 tablet_copy_client.cc:323] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:16.209812 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 57b150e0c7494c95beb04642f466f450 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:16.210366 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 57b150e0c7494c95beb04642f466f450 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-57b150e0c7494c95beb04642f466f450
I20260709 12:13:16.238023 23480 tablet_copy_source_session.cc:215] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.242305 24402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b150e0c7494c95beb04642f466f450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.246613 24403 ts_tablet_manager.cc:933] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:16.260550 24403 tablet_copy_client.cc:323] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:16.262351 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet baab486079784de4933d5f96a0a5321b from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:16.262884 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet baab486079784de4933d5f96a0a5321b from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-baab486079784de4933d5f96a0a5321b
I20260709 12:13:16.267498 24402 tablet_copy_client.cc:806] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.268508 24402 tablet_copy_client.cc:670] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.276064 24402 tablet_copy_client.cc:538] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.282740 23331 tablet_copy_source_session.cc:215] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.286114 24403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet baab486079784de4933d5f96a0a5321b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.287860 24400 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.288987 24400 tablet_bootstrap.cc:492] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:16.289789 24400 ts_tablet_manager.cc:1403] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.088s	user 0.022s	sys 0.004s
I20260709 12:13:16.291407 24402 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:16.293401 24400 raft_consensus.cc:359] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.294346 24400 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:16.295150 24400 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.306010 24403 tablet_copy_client.cc:806] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.306667 24403 tablet_copy_client.cc:670] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.394920 24400 ts_tablet_manager.cc:1434] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.105s	user 0.017s	sys 0.009s
W20260709 12:13:16.395802 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 89394.7us
I20260709 12:13:16.404191 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-d736b5901fbd4de09b83cc09a8d00fd1 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:16.404268 24403 tablet_copy_client.cc:538] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.404773 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-d736b5901fbd4de09b83cc09a8d00fd1 on tablet d736b5901fbd4de09b83cc09a8d00fd1 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:16.411157 24400 ts_tablet_manager.cc:933] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:16.432746 24403 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:16.453109 24400 tablet_copy_client.cc:323] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:16.455233 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 1399c0b792a048cda2bcae40b23432c0 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:16.455850 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 1399c0b792a048cda2bcae40b23432c0 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-1399c0b792a048cda2bcae40b23432c0
I20260709 12:13:16.458770 24402 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.465823 24402 tablet_bootstrap.cc:492] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:16.466888 24402 ts_tablet_manager.cc:1403] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.176s	user 0.009s	sys 0.016s
I20260709 12:13:16.472119 23480 tablet_copy_source_session.cc:215] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:16.475790 24400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1399c0b792a048cda2bcae40b23432c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:16.487519 24015 raft_consensus.cc:1217] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:16.519733 24402 raft_consensus.cc:359] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.520735 24402 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:16.521505 24402 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.529256 24402 ts_tablet_manager.cc:1434] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.001s
I20260709 12:13:16.535998 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-57b150e0c7494c95beb04642f466f450 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:16.536799 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-57b150e0c7494c95beb04642f466f450 on tablet 57b150e0c7494c95beb04642f466f450 with peer 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:13:16.552201 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 130367us
I20260709 12:13:16.558498 24400 tablet_copy_client.cc:806] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:16.559100 24400 tablet_copy_client.cc:670] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:16.572273 24400 tablet_copy_client.cc:538] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:16.576278 24403 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.577805 24403 tablet_bootstrap.cc:492] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:16.578548 24403 ts_tablet_manager.cc:1403] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.146s	user 0.021s	sys 0.003s
I20260709 12:13:16.581725 24403 raft_consensus.cc:359] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.582602 24403 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:16.583289 24403 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.588711 24403 ts_tablet_manager.cc:1434] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:13:16.588711 24400 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:16.593029 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-baab486079784de4933d5f96a0a5321b received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:16.593505 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-baab486079784de4933d5f96a0a5321b on tablet baab486079784de4933d5f96a0a5321b with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:16.596052 24016 raft_consensus.cc:1217] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:16.619673 24400 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:16.620846 24400 tablet_bootstrap.cc:492] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:16.621722 24400 ts_tablet_manager.cc:1403] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.014s
I20260709 12:13:16.622803 24016 raft_consensus.cc:1217] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:16.627976 24400 raft_consensus.cc:359] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.628913 24400 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:16.629987 24400 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:16.635520 24400 ts_tablet_manager.cc:1434] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:13:16.637641 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-1399c0b792a048cda2bcae40b23432c0 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:16.638028 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-1399c0b792a048cda2bcae40b23432c0 on tablet 1399c0b792a048cda2bcae40b23432c0 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:16.731714 24016 raft_consensus.cc:1217] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:16.848296 24382 raft_consensus.cc:1064] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:16.850560 24382 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:16.856609 23462 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:16.857728 23391 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:16.858222 24016 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:16.858299 24382 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:16.859845 24375 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:16.860491 24347 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:16.881067 23391 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:16.882798 23462 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:16.879515 24382 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:16.899032 23218 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:16.899705 24016 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:16.978906 23316 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:16.990465 24016 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:16.992596 24382 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:16.999962 24347 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.011888 24016 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.021049 23462 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.048070 24375 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.051548 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d736b5901fbd4de09b83cc09a8d00fd1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:17.052702 23219 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.066536 24375 raft_consensus.cc:1064] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:17.071045 24375 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.069006 23466 raft_consensus.cc:2955] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.089906 24381 raft_consensus.cc:1064] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:17.092504 24381 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.106094 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet d736b5901fbd4de09b83cc09a8d00fd1 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:17.110675 24424 tablet_replica.cc:333] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:17.115154 24424 raft_consensus.cc:2243] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:17.117355 24424 raft_consensus.cc:2272] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:17.126825 24424 ts_tablet_manager.cc:1916] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:17.127687 24016 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.134019 24015 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.138614 23466 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.139256 23391 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.142865 24396 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.143738 24354 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.146256 24424 ts_tablet_manager.cc:1929] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:17.146800 24424 log.cc:1199] T d736b5901fbd4de09b83cc09a8d00fd1 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/d736b5901fbd4de09b83cc09a8d00fd1
I20260709 12:13:17.147598 24382 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.148696 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet d736b5901fbd4de09b83cc09a8d00fd1 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:17.152563 24382 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.154963 23391 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.157289 23316 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.162680 24379 raft_consensus.cc:1064] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:17.166942 24354 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.166514 24379 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.187692 24375 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.221729 24396 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.224710 24347 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.232534 23392 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.237390 24015 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.241350 24379 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:17.243582 23219 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:17.244831 23316 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.245189 23466 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.257598 23462 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.258772 23217 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:17.278649 23392 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.266906 24016 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:17.286916 24015 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.289695 23392 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.304633 24432 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:17.306780 24432 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:17.321372 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 75411.1us
I20260709 12:13:17.324158 24432 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.331195 24432 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.335567 24015 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.343394 24432 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.346341 24379 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.364228 23316 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.366042 23391 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.351874 24014 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.346534 24016 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.384752 23315 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.392247 23392 raft_consensus.cc:2955] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.392930 23393 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.424369 23219 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:17.432299 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.437920 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 57b150e0c7494c95beb04642f466f450 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:17.447111 24013 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.445845 23391 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.458808 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 57b150e0c7494c95beb04642f466f450 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:17.475135 24438 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:17.477730 24439 tablet_replica.cc:333] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:17.479902 24438 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:17.482375 24439 raft_consensus.cc:2243] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:17.483305 24439 raft_consensus.cc:2272] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:17.486816 24439 ts_tablet_manager.cc:1916] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:17.501736 23462 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:17.508945 24013 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.510490 23321 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:17.514832 24433 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:17.517280 23316 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.530807 24347 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.532807 24433 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:17.534474 24433 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.536443 24014 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.541002 24439 ts_tablet_manager.cc:1929] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:17.541600 24439 log.cc:1199] T 57b150e0c7494c95beb04642f466f450 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/57b150e0c7494c95beb04642f466f450
I20260709 12:13:17.543332 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet baab486079784de4933d5f96a0a5321b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:17.544044 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 57b150e0c7494c95beb04642f466f450 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:17.558338 24014 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.559303 23392 raft_consensus.cc:2955] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.561892 23219 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.577144 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1399c0b792a048cda2bcae40b23432c0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:17.582054 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 1399c0b792a048cda2bcae40b23432c0 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:17.583442 24424 tablet_replica.cc:333] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:17.583014 23218 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:17.584443 24424 raft_consensus.cc:2243] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:17.585235 24424 raft_consensus.cc:2272] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:17.589246 24424 ts_tablet_manager.cc:1916] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:17.599241 23320 raft_consensus.cc:2955] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:17.620270 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet baab486079784de4933d5f96a0a5321b with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:17.634407 24440 tablet_replica.cc:333] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:17.658272 24424 ts_tablet_manager.cc:1929] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:17.658725 24424 log.cc:1199] T 1399c0b792a048cda2bcae40b23432c0 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/1399c0b792a048cda2bcae40b23432c0
I20260709 12:13:17.658958 24440 raft_consensus.cc:2243] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:17.659608 24440 raft_consensus.cc:2272] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:17.659960 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 1399c0b792a048cda2bcae40b23432c0 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:17.662801 24440 ts_tablet_manager.cc:1916] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:17.675560 24440 ts_tablet_manager.cc:1929] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:17.676036 24440 log.cc:1199] T baab486079784de4933d5f96a0a5321b P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/baab486079784de4933d5f96a0a5321b
I20260709 12:13:17.677835 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet baab486079784de4933d5f96a0a5321b (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:18.726754 23321 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:18.733040 23466 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.733808 23391 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:18.734236 23287 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ca56a943ba904b779d7584f94c266344. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:18.735110 24442 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.736030 24198 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.744829 24198 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.746362 23462 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.755029 23391 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.760339 23219 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:18.763852 23321 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:18.785593 23392 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:18.786382 23287 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 746c120ff6e048eaa7b3a03be2b8a91e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:18.785663 23462 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.787721 24198 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.790014 24347 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.799819 24198 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.802690 23392 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.803609 23466 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.823558 23390 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:18.826051 23218 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:18.836033 23362 consensus_peers.cc:597] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 487ee45f93ab48478b7fa939ccf7f658. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:18.836411 23468 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.837800 24205 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.840077 23321 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.842149 24205 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.846119 24411 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.855136 23468 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.856891 23321 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.864586 23390 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:18.865619 23217 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:18.876616 23362 consensus_peers.cc:597] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6161f3a92cf44138f9a68a6f005c4e9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:18.878762 23321 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.880688 23468 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:18.881093 24411 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:18.882329 24205 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:18.894333 24458 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.895902 23466 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.897787 23316 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:18.913409 23217 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:19.154153 24462 ts_tablet_manager.cc:933] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:19.156798 24462 tablet_copy_client.cc:323] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:19.159521 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet ca56a943ba904b779d7584f94c266344 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:19.160125 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet ca56a943ba904b779d7584f94c266344 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-ca56a943ba904b779d7584f94c266344
I20260709 12:13:19.167981 23331 tablet_copy_source_session.cc:215] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:19.171294 24462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca56a943ba904b779d7584f94c266344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:19.183327 24462 tablet_copy_client.cc:806] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:19.184022 24462 tablet_copy_client.cc:670] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:19.188486 24462 tablet_copy_client.cc:538] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:19.197800 24462 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:19.222285 24462 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:19.223158 24462 tablet_bootstrap.cc:492] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:19.223937 24462 ts_tablet_manager.cc:1403] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.008s
I20260709 12:13:19.229266 24462 raft_consensus.cc:359] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.230266 24462 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:19.231369 24462 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.240110 24462 ts_tablet_manager.cc:1434] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:13:19.245069 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ca56a943ba904b779d7584f94c266344 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:19.249133 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ca56a943ba904b779d7584f94c266344 on tablet ca56a943ba904b779d7584f94c266344 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.309571 24462 ts_tablet_manager.cc:933] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:19.311327 24462 tablet_copy_client.cc:323] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:19.313180 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet f6161f3a92cf44138f9a68a6f005c4e9 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:19.313738 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet f6161f3a92cf44138f9a68a6f005c4e9 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-f6161f3a92cf44138f9a68a6f005c4e9
I20260709 12:13:19.320963 23406 tablet_copy_source_session.cc:215] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:19.324577 24462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6161f3a92cf44138f9a68a6f005c4e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:19.339972 24462 tablet_copy_client.cc:806] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:19.340508 24462 tablet_copy_client.cc:670] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:19.344326 24462 tablet_copy_client.cc:538] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:19.354851 24462 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:19.377323 24462 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:19.378334 24462 tablet_bootstrap.cc:492] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:19.379088 24462 ts_tablet_manager.cc:1403] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.024s	user 0.023s	sys 0.000s
I20260709 12:13:19.382128 24462 raft_consensus.cc:359] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.383044 24462 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:19.383826 24462 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.383826 24468 ts_tablet_manager.cc:933] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:19.390395 24462 ts_tablet_manager.cc:1434] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:13:19.393208 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-f6161f3a92cf44138f9a68a6f005c4e9 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:19.393679 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-f6161f3a92cf44138f9a68a6f005c4e9 on tablet f6161f3a92cf44138f9a68a6f005c4e9 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.395303 24468 tablet_copy_client.cc:323] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:19.401146 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 746c120ff6e048eaa7b3a03be2b8a91e from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:19.401747 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 746c120ff6e048eaa7b3a03be2b8a91e from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-746c120ff6e048eaa7b3a03be2b8a91e
I20260709 12:13:19.410362 23331 tablet_copy_source_session.cc:215] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:19.418690 24468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 746c120ff6e048eaa7b3a03be2b8a91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:19.441284 24468 tablet_copy_client.cc:806] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:19.442003 24468 tablet_copy_client.cc:670] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:19.445291 24468 tablet_copy_client.cc:538] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:19.448673 24462 ts_tablet_manager.cc:933] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:19.450434 24462 tablet_copy_client.cc:323] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:19.451880 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 487ee45f93ab48478b7fa939ccf7f658 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:19.452378 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 487ee45f93ab48478b7fa939ccf7f658 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-487ee45f93ab48478b7fa939ccf7f658
I20260709 12:13:19.455031 24468 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:19.464731 23406 tablet_copy_source_session.cc:215] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:19.472883 24462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 487ee45f93ab48478b7fa939ccf7f658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:19.493533 24462 tablet_copy_client.cc:806] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:19.494436 24462 tablet_copy_client.cc:670] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:19.503105 24462 tablet_copy_client.cc:538] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:19.507414 24468 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:19.508479 24468 tablet_bootstrap.cc:492] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:19.509171 24468 ts_tablet_manager.cc:1403] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.009s
I20260709 12:13:19.510281 24462 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:19.512339 24468 raft_consensus.cc:359] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.513337 24468 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:19.514173 24468 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.516866 24468 ts_tablet_manager.cc:1434] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:13:19.527740 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-746c120ff6e048eaa7b3a03be2b8a91e received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:19.528175 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-746c120ff6e048eaa7b3a03be2b8a91e on tablet 746c120ff6e048eaa7b3a03be2b8a91e with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.540170 24462 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:19.541090 24462 tablet_bootstrap.cc:492] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:19.541810 24462 ts_tablet_manager.cc:1403] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.004s
I20260709 12:13:19.544044 24462 raft_consensus.cc:359] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.544625 24462 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:19.545092 24462 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:19.547076 24462 ts_tablet_manager.cc:1434] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:13:19.549265 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-487ee45f93ab48478b7fa939ccf7f658 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:19.549686 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-487ee45f93ab48478b7fa939ccf7f658 on tablet 487ee45f93ab48478b7fa939ccf7f658 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.614392 24014 raft_consensus.cc:1217] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:19.681668 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:19.775897 24014 raft_consensus.cc:1217] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:19.813966 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:19.814719 23316 raft_consensus.cc:606] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:19.821760 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:19.822371 23316 raft_consensus.cc:606] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:19.840677 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:19.841285 23390 raft_consensus.cc:606] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.844599 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:19.845189 23316 raft_consensus.cc:606] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:19.847010 24198 raft_consensus.cc:993] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:19.847455 24198 raft_consensus.cc:1081] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:19.848191 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:19.848781 23316 raft_consensus.cc:606] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:19.849880 24014 raft_consensus.cc:1217] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:19.850116 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:19.850695 23466 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:19.851027 23466 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.855119 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:19.855746 23316 raft_consensus.cc:606] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:19.857407 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:13:19.859113 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:19.860672 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:19.870487 23466 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:19.875299 23466 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:19.875504 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:19.875566 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:19.876297 23390 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.876504 23316 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:19.876796 23316 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:19.877544 23316 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:19.878726 23316 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.881742 24411 raft_consensus.cc:993] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:19.882234 24411 raft_consensus.cc:1081] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:19.883231 23390 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:19.884030 24013 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53950
I20260709 12:13:19.884604 24013 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:19.884958 24013 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.885548 23316 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:19.886967 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:19.887751 24396 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:19.888159 24396 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:19.888890 24396 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:19.894384 24013 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:19.898439 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:19.899003 23321 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.899124 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:19.899760 23390 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:19.900105 23390 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:19.900985 23390 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:19.902129 23390 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:19.905781 23321 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:19.907021 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:19.907980 24465 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:19.909102 24465 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:19.910091 24465 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:19.914327 24013 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:19.926925 23218 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:19.929886 23217 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:19.937923 23390 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:19.999992 24013 raft_consensus.cc:1217] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:20.036810 24454 raft_consensus.cc:1064] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:13:20.037284 24454 raft_consensus.cc:1068] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Unable to promote non-voter 54640ab8abeb4c949426d66c7e5aed87: Service unavailable: leader transfer in progress
I20260709 12:13:20.041249 24454 raft_consensus.cc:993] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.041689 24454 raft_consensus.cc:1081] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.046370 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:20.046892 23466 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:20.047224 23466 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.052271 23466 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.053892 23466 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:20.054950 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:20.055256 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:20.055552 23390 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.055835 23321 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:20.056169 23321 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:20.056795 23321 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.057698 23321 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.061957 23390 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.063035 23321 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.063315 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:20.064078 24396 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:20.064546 24396 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:20.065371 24396 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.084045 24198 raft_consensus.cc:993] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.084510 24442 raft_consensus.cc:1081] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.086262 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:20.086799 23466 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:20.087152 23466 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.086429 23218 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.096915 23466 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:20.099570 23466 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:20.100660 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:20.101249 23390 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.101763 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:20.102283 23321 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:20.102540 23321 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:20.103232 23321 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:20.104480 23321 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.107671 23390 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.108774 23436 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:20.109556 24433 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:20.110011 24433 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:20.110852 23321 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.110895 24433 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:20.122030 23218 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:20.132638 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:20.133226 23466 raft_consensus.cc:606] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership
I20260709 12:13:20.136395 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:20.136802 23466 raft_consensus.cc:606] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:13:20.137790 23206 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for ca56a943ba904b779d7584f94c266344 already in progress [suppressed 5 similar messages]
I20260709 12:13:20.140082 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:20.140496 23466 raft_consensus.cc:606] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership
W20260709 12:13:20.144168 23267 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:13:20.144658 23267 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for ca56a943ba904b779d7584f94c266344 already in progress
W20260709 12:13:20.145035 23267 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for ca56a943ba904b779d7584f94c266344 already in progress
I20260709 12:13:20.203557 24454 raft_consensus.cc:993] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:20.204137 24442 raft_consensus.cc:1081] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:20.207829 24013 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:20.208498 24013 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:20.208920 24013 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.220443 24013 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.223817 24013 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:20.224035 24442 raft_consensus.cc:993] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.225646 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:20.226009 24442 raft_consensus.cc:1081] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:20.226382 23321 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:20.226893 23321 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:20.227686 23321 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.228699 23321 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.230285 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:20.230886 23466 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.235515 23321 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:20.236109 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:20.236670 23462 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:20.237010 23462 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.237627 23466 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:20.238644 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:20.239359 24465 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:20.239734 24465 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:20.240361 24465 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.254289 23462 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.257505 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:20.257993 23392 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.259349 23218 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.263509 23392 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.264957 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:20.265559 23316 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:20.265838 23316 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:20.265844 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:20.266815 24396 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:20.266739 23316 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.267305 24396 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:20.267851 23316 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:20.268150 24396 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:20.275003 23316 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:20.277519 23462 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:20.286530 23392 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.290480 24347 raft_consensus.cc:1064] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:20.292768 24347 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.293505 23218 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.297773 24487 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:20.344362 23462 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.349414 23390 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.349506 24013 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.376271 24411 raft_consensus.cc:1064] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:20.379753 23316 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.378903 24411 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.386370 24492 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:20.392378 24492 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:20.397610 24492 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:20.407740 23316 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:20.408097 24433 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:20.409051 24465 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:20.450834 24492 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.458256 23392 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.460289 24487 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:20.467342 23466 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.477365 23391 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:20.482999 24477 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:20.502210 23321 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.501714 24013 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.512018 23390 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.523664 23219 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.526176 23321 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.527750 24487 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:20.526213 24014 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.531594 24441 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:20.570047 23466 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.605580 24016 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Advancing to term 2
I20260709 12:13:20.628234 24016 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:20.641880 23315 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:20.641279 24396 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:13:20.685993 24441 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:13:20.708597 24511 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:13:20.711017 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 72351.1us
W20260709 12:13:20.720953 24512 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:13:20.722450 24433 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:20.724536 23315 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:20.727424 23392 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.728701 23390 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:13:20.746658 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 75518.7us
I20260709 12:13:20.751863 24465 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:13:20.769677 24520 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:20.771258 24411 raft_consensus.cc:1064] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:20.777940 24411 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.795651 23316 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:20.797596 24519 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
W20260709 12:13:20.885844 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 161378us
I20260709 12:13:20.895210 24522 raft_consensus.cc:1064] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:13:20.895694 24522 raft_consensus.cc:1068] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Unable to promote non-voter 54640ab8abeb4c949426d66c7e5aed87: Service unavailable: leader transfer in progress
I20260709 12:13:20.898569 24514 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.901672 23468 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:20.903033 24465 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:20.906949 23468 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:20.912819 23315 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:20.923949 24016 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:20.932745 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 167588us
I20260709 12:13:20.937779 24014 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:20.940465 23468 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:20.944801 23463 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.952098 24014 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:20.955483 24529 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:20.956358 24411 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:20.980612 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:20.982056 23219 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.987501 24521 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:21.041630 24519 raft_consensus.cc:993] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:21.042101 24519 raft_consensus.cc:1081] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:21.050272 23321 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:21.052424 24519 raft_consensus.cc:1064] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:13:21.052839 24519 raft_consensus.cc:1068] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Unable to promote non-voter 54640ab8abeb4c949426d66c7e5aed87: Service unavailable: leader transfer in progress
I20260709 12:13:21.053968 23321 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.062762 24442 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:21.063009 24529 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.070667 23318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42792
I20260709 12:13:21.071321 23318 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.071676 23318 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:21.066404 24016 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.072891 24523 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:21.080077 23318 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:21.086055 23318 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.111339 23394 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.114835 24411 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:21.124295 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:21.124855 23393 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:21.125325 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.125900 23462 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:21.126200 23462 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:21.127171 23462 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:21.131105 23462 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:21.134060 24491 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.136616 24016 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.138024 24411 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.146900 23462 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:21.148102 23393 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:21.147848 24508 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.151974 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.157332 23315 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.159518 24526 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:21.160053 24526 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.161623 24016 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.160967 24526 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:21.151849 24013 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.171475 23466 raft_consensus.cc:2955] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.170651 24458 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:13:21.173345 23319 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.181537 23468 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.198359 23214 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.215080 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f6161f3a92cf44138f9a68a6f005c4e9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.218866 23315 raft_consensus.cc:2955] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.229624 23393 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.230358 23217 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.234028 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 746c120ff6e048eaa7b3a03be2b8a91e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.245850 24016 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.248334 24508 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.251580 23466 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:21.253032 24508 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.260609 24508 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.264219 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet f6161f3a92cf44138f9a68a6f005c4e9 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:21.264555 23466 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.269461 24016 raft_consensus.cc:2955] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.276885 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 487ee45f93ab48478b7fa939ccf7f658 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:21.276312 23214 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.280153 24544 tablet_replica.cc:333] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:21.281260 24544 raft_consensus.cc:2243] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.282114 24544 raft_consensus.cc:2272] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.307058 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 746c120ff6e048eaa7b3a03be2b8a91e with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:21.318369 24545 tablet_replica.cc:333] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:21.319546 24545 raft_consensus.cc:2243] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.320250 24545 raft_consensus.cc:2272] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.306555 23216 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.325127 24544 ts_tablet_manager.cc:1916] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.319988 23218 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.338872 24545 ts_tablet_manager.cc:1916] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.351681 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 487ee45f93ab48478b7fa939ccf7f658 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:21.373847 24544 ts_tablet_manager.cc:1929] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:21.374583 24544 log.cc:1199] T f6161f3a92cf44138f9a68a6f005c4e9 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/f6161f3a92cf44138f9a68a6f005c4e9
I20260709 12:13:21.377576 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet f6161f3a92cf44138f9a68a6f005c4e9 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:21.373394 24548 tablet_replica.cc:333] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:21.382507 24548 raft_consensus.cc:2243] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:21.385797 24548 raft_consensus.cc:2272] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:21.390384 24548 ts_tablet_manager.cc:1916] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:21.394588 24545 ts_tablet_manager.cc:1929] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:21.395306 24545 log.cc:1199] T 746c120ff6e048eaa7b3a03be2b8a91e P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/746c120ff6e048eaa7b3a03be2b8a91e
I20260709 12:13:21.398176 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 746c120ff6e048eaa7b3a03be2b8a91e (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:21.399780 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:21.400378 23466 raft_consensus.cc:606] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:21.405468 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:21.407310 23393 raft_consensus.cc:606] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:21.410483 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:21.411161 23315 raft_consensus.cc:606] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:21.414681 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:21.414706 24548 ts_tablet_manager.cc:1929] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:21.415441 23315 raft_consensus.cc:606] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:21.415699 24548 log.cc:1199] T 487ee45f93ab48478b7fa939ccf7f658 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/487ee45f93ab48478b7fa939ccf7f658
I20260709 12:13:21.417423 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 487ee45f93ab48478b7fa939ccf7f658 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:21.418124 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:21.418745 23315 raft_consensus.cc:606] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:21.421015 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:21.421476 23315 raft_consensus.cc:606] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:21.423436 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:21.423816 23315 raft_consensus.cc:606] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:21.424832 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:13:21.426198 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:21.426784 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:21.513752 24491 raft_consensus.cc:993] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.514273 24526 raft_consensus.cc:1081] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.515650 24016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:21.516095 24016 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.516381 24016 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.523030 24016 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.524430 24016 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.525388 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.525537 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:21.526026 23462 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.526129 23315 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.526500 23315 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.527171 23315 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.528304 23315 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.531323 23462 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.532776 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.533629 24536 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.534723 23315 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.537060 24536 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:21.538048 24536 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.547003 23216 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.572706 23462 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:21.574654 24526 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.589385 24016 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Advancing to term 3
I20260709 12:13:21.594743 23393 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:21.596561 24442 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.604239 24016 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:13:21.607358 24442 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:21.623179 24526 raft_consensus.cc:1064] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:21.625890 24526 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.630260 24523 raft_consensus.cc:993] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:21.630743 24491 raft_consensus.cc:1081] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:21.632149 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:21.632733 23462 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.633229 23466 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:21.633607 23462 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.634182 23394 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:21.635360 24526 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.636494 24523 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.639941 23462 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.641108 24016 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:21.642084 24514 raft_consensus.cc:993] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:21.642666 24524 raft_consensus.cc:1081] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:21.643561 23462 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:21.644328 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:21.644492 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:21.644884 23315 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.645009 23394 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.645272 23315 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.651450 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:21.652118 23321 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.651736 23315 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:13:21.652379 24491 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:21.652462 23321 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.653582 23321 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.654016 23315 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.657639 23394 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:21.655349 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:21.654821 23321 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.655519 24523 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.655023 23393 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.660857 23390 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.661698 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.661814 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.668458 23466 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.668936 24396 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.669396 24396 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:21.670037 23321 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:21.670154 24396 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.663177 24016 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.667411 23390 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:21.681290 23390 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:13:21.687151 23390 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.688076 23216 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.692920 23462 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:21.699739 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.714738 23466 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:21.730803 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.731233 24491 raft_consensus.cc:993] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.731631 24442 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.731741 24491 raft_consensus.cc:1081] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.732182 24442 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.733049 24442 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } }
I20260709 12:13:21.697052 23390 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:21.735569 24014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:21.736130 24014 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.736455 24014 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.756735 24014 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.759860 24014 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:21.759992 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:21.760627 23321 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.760946 23321 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.761511 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:21.761632 23321 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.762326 23390 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.762864 23321 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.766765 23217 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.768457 23390 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.769582 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:21.770362 24536 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.770826 24536 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:21.769161 23321 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.771693 24536 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.774925 24396 raft_consensus.cc:993] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.775449 24396 raft_consensus.cc:1081] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:21.777657 24016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53926
I20260709 12:13:21.778224 24016 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.778584 24016 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.782284 23217 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.786244 24016 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.789373 24016 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.791345 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.791935 23462 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.792264 23462 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:21.792843 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:21.792958 23462 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.793459 23390 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.794203 23462 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.801741 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.802819 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.803697 24536 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.804222 23390 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:21.804924 24536 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:21.808238 24536 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:21.819304 23217 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:21.864149 24526 raft_consensus.cc:993] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:21.864704 24442 raft_consensus.cc:1081] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:21.866093 23390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:21.866496 23390 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.866825 23390 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.871129 23390 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.873492 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:21.874053 23315 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.874351 23315 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.874752 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.875104 23315 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.875630 23462 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.876235 23315 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.880975 23390 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.886575 23462 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:21.886579 23315 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:21.887900 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.888682 24514 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.889149 24514 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:21.890091 24514 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.900219 23217 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.946538 24526 raft_consensus.cc:993] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:21.947024 24526 raft_consensus.cc:1081] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:21.948402 23393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:21.948889 23393 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:21.949213 23393 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.952464 23462 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:21.953876 24536 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.957587 23393 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.960330 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:21.960852 23466 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.961741 23393 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:21.964280 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:21.964862 23315 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:21.965181 23315 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:21.965937 23315 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.967053 23315 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:21.967581 23466 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:21.968741 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:21.969596 24524 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:21.970093 24524 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:21.970921 24524 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:21.977008 23321 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:21.980724 24536 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:21.988750 23217 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:21.989687 23315 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:22.148764 23390 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.149906 24513 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.166925 23321 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.168161 24522 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.252806 23470 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.254381 24198 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.264719 23315 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:22.267962 23390 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.269534 24563 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.272338 24016 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:22.273140 23393 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:22.274824 24198 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:22.276208 24198 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:22.290832 24198 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:22.295532 23315 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:22.296928 24536 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:22.298727 23470 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:22.299111 23390 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:22.301450 24536 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:22.306841 24016 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:22.320173 24536 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:22.321404 23394 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:22.327610 23393 raft_consensus.cc:2955] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:22.335199 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ca56a943ba904b779d7584f94c266344 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:22.339052 24536 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:22.364534 23218 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 5 to 6, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:22.431914 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet ca56a943ba904b779d7584f94c266344 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:22.464021 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:22.465445 23321 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.467072 24577 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.478071 24579 tablet_replica.cc:333] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:22.479104 24579 raft_consensus.cc:2243] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:22.479871 24579 raft_consensus.cc:2272] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:22.483420 24579 ts_tablet_manager.cc:1916] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:13:22.483848 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 132819us
I20260709 12:13:22.485416 23470 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.488721 24524 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.509965 23321 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.511365 24574 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.519794 24579 ts_tablet_manager.cc:1929] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:13:22.520334 24579 log.cc:1199] T ca56a943ba904b779d7584f94c266344 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/ca56a943ba904b779d7584f94c266344
I20260709 12:13:22.522089 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet ca56a943ba904b779d7584f94c266344 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:22.556500 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:22.557124 23315 raft_consensus.cc:606] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:22.576892 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:22.577582 23315 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:22.580912 23470 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:22.581794 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:22.594384 23466 raft_consensus.cc:606] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:22.596009 24577 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.597590 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:22.598624 23315 raft_consensus.cc:606] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:22.605310 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:22.605994 23315 raft_consensus.cc:606] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:22.608748 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:22.609894 23315 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:22.617408 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de7238eb389343cd954cb756bed5925f"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:22.618001 23390 raft_consensus.cc:606] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:22.619374 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:13:22.621028 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:22.621795 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:22.667008 23321 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:22.667467 24526 raft_consensus.cc:993] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:22.670116 24526 raft_consensus.cc:1081] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:22.672196 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:22.672721 23466 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:22.673041 23466 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.680114 23466 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.682523 23466 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:22.682689 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:22.682888 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:22.683306 23390 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.683614 23315 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:22.684011 23315 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:22.685127 23466 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:22.685928 23393 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:22.686094 23315 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.687280 23315 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.687170 24198 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:13:22.688287 23287 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c192a749705545f298d61bbb8ccb5665. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.689621 23390 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:22.688308 24526 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:13:22.690738 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:22.692790 24513 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:22.693262 24513 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:22.694056 24513 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.694752 23315 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:22.700738 24526 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.707767 23466 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.722553 23390 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.730425 23321 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:22.731199 23218 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.737663 24513 raft_consensus.cc:993] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:22.738147 24513 raft_consensus.cc:1081] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:22.738363 23217 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:22.743018 23470 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:22.743417 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42792
W20260709 12:13:22.743659 23287 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1bca0d75a0cb485ebd79c619c3bf8341. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.743964 23315 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:22.744341 23315 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.745491 23393 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:22.746255 24442 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:22.747558 24442 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:22.754496 24523 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.762907 23315 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.765877 23315 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:22.765934 23470 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.770455 23462 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:22.785856 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:22.786500 23390 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:22.789577 23436 consensus_peers.cc:597] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d83783f67774af1b007547d0aca8da1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.792428 23393 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:22.793893 24522 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:22.794534 23390 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:22.800289 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:22.801023 23462 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:22.801707 23462 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:22.801265 23393 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.802625 23462 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.803833 23462 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.803000 23214 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:22.814728 24396 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.818434 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:22.819830 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:22.821796 24589 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:22.822268 24589 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:22.823140 24589 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.823856 23321 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:22.825867 23390 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.833520 24522 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:22.840148 24442 raft_consensus.cc:993] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:22.840596 24442 raft_consensus.cc:1081] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:22.842962 23393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:22.843526 23393 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:22.843441 23315 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:22.844007 23393 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.848543 23216 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:22.858043 24563 raft_consensus.cc:993] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:22.858819 24563 raft_consensus.cc:1081] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:22.862020 24014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:22.862685 24014 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:22.863058 24014 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:22.872252 24014 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:22.872771 23393 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.890147 23393 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:22.890887 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:22.892360 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:22.892645 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:22.893110 23319 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:22.893702 23319 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:22.875011 23321 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.894742 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:22.894542 23319 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.895298 23466 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:22.895658 23319 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.882136 23470 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:22.886994 23287 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9d236b23181b45e5a16e91a3b6a30058. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:22.903533 24442 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:22.888360 23390 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:22.909199 24442 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:22.893260 23462 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:22.919489 23466 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:22.920178 23468 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:22.923174 24442 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:22.924444 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:22.925509 24572 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:22.925976 24572 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:22.927326 24572 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:22.891451 23315 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:22.930709 23315 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:22.932683 24590 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.931788 23315 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:22.937350 23319 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:22.938256 23315 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:22.943727 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:22.944584 24577 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:22.945328 24014 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:22.946655 24577 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:22.947598 24577 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:22.947467 23470 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:22.955421 23315 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:22.972353 23462 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
W20260709 12:13:22.957935 24523 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:13:23.009300 23216 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:23.019289 24198 raft_consensus.cc:993] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.019915 24198 raft_consensus.cc:1081] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.026194 24016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:23.026896 24016 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:23.027325 24016 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:23.042696 24016 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.055956 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:23.056730 23319 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:23.057096 23319 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:23.058378 23319 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.060668 23319 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:23.068030 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:23.070590 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:23.072537 23468 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:23.071369 24597 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:13:23.080914 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:23.087937 24575 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:23.093667 24563 raft_consensus.cc:993] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.094442 24563 raft_consensus.cc:1081] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.102312 24576 raft_consensus.cc:993] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.103024 24576 raft_consensus.cc:1081] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:23.106287 24575 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:23.107511 24575 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.110411 24014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:23.117149 24014 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:23.117580 24014 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:23.120843 23393 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.122478 23218 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:23.123421 24396 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:23.131518 24014 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.123806 24015 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de7238eb389343cd954cb756bed5925f"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53950
I20260709 12:13:23.132957 24015 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:23.133314 24015 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:23.143383 23393 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:23.126303 23468 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:23.126685 24016 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:23.150111 24015 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.152256 24015 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:23.154325 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:23.154925 23468 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:23.159461 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:23.159091 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:23.160051 23394 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:23.160368 23394 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:23.161103 23394 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.162742 23394 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:23.166283 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:23.166898 23319 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:23.167203 23319 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:23.153060 23216 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:23.170972 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:23.171543 23392 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:23.168996 23468 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:23.169973 24014 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:23.170331 23315 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.175208 23218 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:23.178875 23392 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:23.169682 23319 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.189415 23319 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:23.181907 23394 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:23.193915 24513 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:23.197489 23319 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:23.184031 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:23.198889 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:23.199975 24572 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:23.200470 24572 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:23.201303 24572 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.201210 24573 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:23.213958 24573 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:23.214941 24573 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:23.277107 23214 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:23.281183 24608 ts_tablet_manager.cc:933] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:23.281473 23214 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:23.291970 24608 tablet_copy_client.cc:323] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:23.294727 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet c192a749705545f298d61bbb8ccb5665 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:23.295274 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet c192a749705545f298d61bbb8ccb5665 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-c192a749705545f298d61bbb8ccb5665
I20260709 12:13:23.303874 23331 tablet_copy_source_session.cc:215] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:23.307845 24608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c192a749705545f298d61bbb8ccb5665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:23.320190 24615 ts_tablet_manager.cc:933] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:23.322239 24614 ts_tablet_manager.cc:933] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:23.330325 24615 tablet_copy_client.cc:323] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:23.333578 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 1bca0d75a0cb485ebd79c619c3bf8341 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:23.334354 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 1bca0d75a0cb485ebd79c619c3bf8341 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-1bca0d75a0cb485ebd79c619c3bf8341
I20260709 12:13:23.334546 24608 tablet_copy_client.cc:806] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:23.335167 24608 tablet_copy_client.cc:670] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:23.338124 24614 tablet_copy_client.cc:323] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:23.339362 24608 tablet_copy_client.cc:538] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:23.339861 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 6d83783f67774af1b007547d0aca8da1 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:23.340422 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 6d83783f67774af1b007547d0aca8da1 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-6d83783f67774af1b007547d0aca8da1
I20260709 12:13:23.344404 23331 tablet_copy_source_session.cc:215] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:23.349169 24615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bca0d75a0cb485ebd79c619c3bf8341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:23.349696 23480 tablet_copy_source_session.cc:215] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:23.353535 24614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d83783f67774af1b007547d0aca8da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:23.357162 23468 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.360877 23315 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.360934 24590 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:23.362562 24524 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:23.377285 23462 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:23.379282 24572 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:23.349212 24608 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:23.390105 23390 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.420962 24618 ts_tablet_manager.cc:933] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:23.438428 23468 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:23.459260 24577 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:23.410262 24602 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:13:23.465694 23315 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:23.468520 24572 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:23.508046 24618 tablet_copy_client.cc:323] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:23.528826 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 9d236b23181b45e5a16e91a3b6a30058 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:23.529469 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 9d236b23181b45e5a16e91a3b6a30058 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-9d236b23181b45e5a16e91a3b6a30058
I20260709 12:13:23.516839 24614 tablet_copy_client.cc:806] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:23.593811 24614 tablet_copy_client.cc:670] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:23.674657 23466 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:23.688838 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:23.716823 24615 tablet_copy_client.cc:806] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:23.717527 24615 tablet_copy_client.cc:670] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:23.755419 23315 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:23.746614 24615 tablet_copy_client.cc:538] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:13:23.764817 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 137089us
I20260709 12:13:23.790064 23331 tablet_copy_source_session.cc:215] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:23.804230 24573 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:23.805284 24618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d236b23181b45e5a16e91a3b6a30058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:23.814265 24608 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:23.815456 24608 tablet_bootstrap.cc:492] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:23.820586 24608 ts_tablet_manager.cc:1403] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.472s	user 0.031s	sys 0.024s
I20260709 12:13:23.814436 24615 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:23.838343 23468 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:23.864873 23392 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:23.839594 24631 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:13:23.854390 24614 tablet_copy_client.cc:538] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:23.884133 24608 raft_consensus.cc:359] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:23.874931 24572 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:13:23.912128 24608 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:23.920750 24608 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:23.923262 24608 ts_tablet_manager.cc:1434] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.001s
I20260709 12:13:23.931767 23390 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:23.937095 23316 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:13:23.940866 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 83221.8us
I20260709 12:13:23.941385 24573 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:13:23.916524 24634 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:13:23.960570 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:23.961272 23390 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:23.963795 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c192a749705545f298d61bbb8ccb5665 received from {username='slave'} at 127.0.0.1:43746
W20260709 12:13:24.001097 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53390.6us
W20260709 12:13:24.007903 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60423.5us
I20260709 12:13:24.005321 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c192a749705545f298d61bbb8ccb5665 on tablet c192a749705545f298d61bbb8ccb5665 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:24.027311 24575 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:24.028180 24618 tablet_copy_client.cc:806] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:24.028805 24618 tablet_copy_client.cc:670] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:24.041792 24618 tablet_copy_client.cc:538] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:24.053373 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:24.054257 23466 raft_consensus.cc:606] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:24.081061 24614 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
W20260709 12:13:24.081310 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49121us
I20260709 12:13:24.104295 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:13:24.119470 24015 raft_consensus.cc:1217] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:24.122474 24618 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:24.141826 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:24.142705 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:24.160084 24640 raft_consensus.cc:993] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:24.160624 24640 raft_consensus.cc:1081] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:24.195938 23390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:13:24.196660 23390 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:24.197044 23390 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:24.213647 23390 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.215600 23390 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:24.220371 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:24.220930 23468 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:24.221204 23468 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:24.221876 23468 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.222987 23468 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:24.228091 24615 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.229106 24615 tablet_bootstrap.cc:492] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:24.229830 24615 ts_tablet_manager.cc:1403] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.416s	user 0.042s	sys 0.016s
I20260709 12:13:24.233199 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:24.233367 24615 raft_consensus.cc:359] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.234339 24615 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:24.235069 24615 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.234505 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:24.236482 24628 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:24.237113 24628 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:24.237985 24628 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.244298 24615 ts_tablet_manager.cc:1434] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:13:24.246573 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:24.247200 23317 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:24.256148 24574 raft_consensus.cc:993] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:24.256695 24574 raft_consensus.cc:1081] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:24.259456 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:24.260061 23315 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:24.260394 23315 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:24.274302 23317 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:24.290366 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-1bca0d75a0cb485ebd79c619c3bf8341 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:24.290846 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-1bca0d75a0cb485ebd79c619c3bf8341 on tablet 1bca0d75a0cb485ebd79c619c3bf8341 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:24.298602 24618 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.298432 23315 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.299564 24618 tablet_bootstrap.cc:492] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:24.300272 24618 ts_tablet_manager.cc:1403] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.178s	user 0.014s	sys 0.018s
I20260709 12:13:24.300812 23315 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:24.304991 24618 raft_consensus.cc:359] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.305903 24618 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:24.306607 24618 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.312453 24618 ts_tablet_manager.cc:1434] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:13:24.314069 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-9d236b23181b45e5a16e91a3b6a30058 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:24.314481 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-9d236b23181b45e5a16e91a3b6a30058 on tablet 9d236b23181b45e5a16e91a3b6a30058 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:24.319825 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:24.320369 23394 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:24.320647 23394 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:24.321364 23394 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.322408 23394 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:24.332903 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:24.333609 23468 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:24.345661 23214 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:24.349550 23394 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:24.350984 24614 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:24.350817 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:24.351864 24526 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:24.352006 24614 tablet_bootstrap.cc:492] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:24.352387 24526 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:24.352814 24614 ts_tablet_manager.cc:1403] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.272s	user 0.023s	sys 0.014s
I20260709 12:13:24.353260 24526 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:24.355960 24614 raft_consensus.cc:359] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.356839 24614 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:24.358064 24614 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:24.364965 24614 ts_tablet_manager.cc:1434] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:13:24.366811 23468 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:24.370434 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-6d83783f67774af1b007547d0aca8da1 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:24.370880 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-6d83783f67774af1b007547d0aca8da1 on tablet 6d83783f67774af1b007547d0aca8da1 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:24.390626 23218 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:24.442586 24014 raft_consensus.cc:1217] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:24.684347 23470 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:24.685688 24574 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:24.694967 23315 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:24.696305 24628 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:24.709748 24014 raft_consensus.cc:1217] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:24.709883 24523 raft_consensus.cc:1064] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:24.732023 24523 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:24.748658 23394 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:24.755699 24014 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:24.757381 23470 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:24.753322 24523 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:13:24.769742 24602 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:24.770967 24638 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:24.771827 23394 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:24.777489 23468 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:24.780556 24014 raft_consensus.cc:1217] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:24.794345 24563 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:24.795190 24602 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:24.853865 24523 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:24.880635 23463 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:24.911068 24015 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:24.921758 23214 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:24.937788 23394 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:24.940315 24198 raft_consensus.cc:1064] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:24.944057 24198 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.006338 23463 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.011581 24015 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.015764 24198 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.018823 24198 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.025560 23390 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.029734 24198 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.050962 23315 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.051899 24198 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.054116 23390 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.058395 24014 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:25.060556 23463 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:25.061666 24563 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.062510 24638 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.079200 24523 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.082242 24014 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.082582 24015 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.092422 23216 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.097020 23466 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.109659 23470 raft_consensus.cc:2955] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.120316 23317 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.123000 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c192a749705545f298d61bbb8ccb5665 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.130028 23218 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.146437 24014 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:25.146566 23390 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:25.147888 24602 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.150060 24602 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.166163 24602 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.170765 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet c192a749705545f298d61bbb8ccb5665 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:25.174048 24682 tablet_replica.cc:333] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:25.175297 24682 raft_consensus.cc:2243] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.176115 24682 raft_consensus.cc:2272] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.179533 23391 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.180094 24014 raft_consensus.cc:2955] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.186443 24682 ts_tablet_manager.cc:1916] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.200693 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1bca0d75a0cb485ebd79c619c3bf8341 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.199079 23216 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.219542 24682 ts_tablet_manager.cc:1929] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:25.220878 24682 log.cc:1199] T c192a749705545f298d61bbb8ccb5665 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/c192a749705545f298d61bbb8ccb5665
I20260709 12:13:25.257299 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet c192a749705545f298d61bbb8ccb5665 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:25.275234 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 1bca0d75a0cb485ebd79c619c3bf8341 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:25.294785 24684 tablet_replica.cc:333] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:25.298170 24684 raft_consensus.cc:2243] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.309262 24684 raft_consensus.cc:2272] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.315240 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:25.324865 24563 raft_consensus.cc:1064] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:25.332449 24684 ts_tablet_manager.cc:1916] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.352927 24563 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.365006 24684 ts_tablet_manager.cc:1929] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:25.365852 24684 log.cc:1199] T 1bca0d75a0cb485ebd79c619c3bf8341 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/1bca0d75a0cb485ebd79c619c3bf8341
I20260709 12:13:25.390329 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 1bca0d75a0cb485ebd79c619c3bf8341 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:25.415372 24649 raft_consensus.cc:1064] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:25.444799 24015 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:25.446398 23390 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:25.449883 24526 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:25.457840 24198 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:25.459174 23470 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:13:25.451875 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77183.4us
I20260709 12:13:25.480391 24638 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:25.498353 24649 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.529001 24526 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.531939 24015 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.530740 23391 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.546662 23466 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.564441 23218 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.569684 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:25.576745 23393 raft_consensus.cc:606] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:25.579329 24014 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.581027 24649 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.582337 23393 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.583534 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:25.584821 24649 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.585368 23315 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:25.589097 23463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:25.589695 23463 raft_consensus.cc:606] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:25.591041 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:25.593015 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:25.596302 23315 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:25.614687 24686 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.621316 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:25.620189 24015 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.637390 23319 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.644447 23393 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.656224 23218 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.665670 24686 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.670322 23468 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:25.671100 24014 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:25.677879 23393 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:25.683459 24649 raft_consensus.cc:993] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:25.683955 24649 raft_consensus.cc:1081] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:25.684319 23393 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:25.685119 24015 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:25.685591 23391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:13:25.686188 23391 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:25.686514 23391 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.686625 24681 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:25.692101 24649 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:25.696311 24396 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.706954 24014 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.706947 23393 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.721776 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6d83783f67774af1b007547d0aca8da1 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.724756 24198 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:25.726825 24574 raft_consensus.cc:993] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:25.727324 24574 raft_consensus.cc:1081] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:25.731412 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:25.731978 23315 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:25.732362 23315 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.745157 23218 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.753813 24602 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:25.780377 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 6d83783f67774af1b007547d0aca8da1 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:25.786456 23391 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.796190 24198 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.806954 23391 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:25.807758 23390 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.811631 23315 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.815608 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9d236b23181b45e5a16e91a3b6a30058 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:25.824100 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:25.824816 23468 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:25.825083 23468 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:25.825909 23468 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.825601 23316 raft_consensus.cc:2955] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.827212 23468 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.822106 24014 raft_consensus.cc:2955] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:25.833061 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:25.833711 23317 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.834489 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:25.835350 23391 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:25.835582 23315 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:25.835750 23391 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:25.839021 23391 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.840262 23391 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.843959 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:25.844551 23466 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:25.849603 23391 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:25.851104 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:25.854388 24526 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:25.854854 24526 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:25.854611 23217 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 3 to 4, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 1 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:25.855628 24526 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.858247 24697 tablet_replica.cc:333] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:25.859432 24697 raft_consensus.cc:2243] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.860033 24697 raft_consensus.cc:2272] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.864950 24697 ts_tablet_manager.cc:1916] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.871575 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 9d236b23181b45e5a16e91a3b6a30058 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:25.872555 24684 tablet_replica.cc:333] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:25.873642 24684 raft_consensus.cc:2243] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:25.874435 24684 raft_consensus.cc:2272] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:25.878311 24684 ts_tablet_manager.cc:1916] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:25.888377 23317 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:25.894874 23468 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:25.894583 23466 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:25.897120 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:25.898061 24574 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:25.899025 24574 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:25.899868 24574 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:25.907032 23214 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:25.912397 24697 ts_tablet_manager.cc:1929] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:25.912959 24697 log.cc:1199] T 6d83783f67774af1b007547d0aca8da1 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/6d83783f67774af1b007547d0aca8da1
I20260709 12:13:25.913990 24684 ts_tablet_manager.cc:1929] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:25.914445 24684 log.cc:1199] T 9d236b23181b45e5a16e91a3b6a30058 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/9d236b23181b45e5a16e91a3b6a30058
I20260709 12:13:25.915288 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 6d83783f67774af1b007547d0aca8da1 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:25.917590 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 9d236b23181b45e5a16e91a3b6a30058 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:25.923763 23214 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.067871 24198 raft_consensus.cc:993] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:26.068394 24526 raft_consensus.cc:1081] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:26.069911 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:26.070474 23466 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:26.070761 23466 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:26.075639 23466 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:26.077229 23466 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:26.078025 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:26.078244 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:26.078676 23317 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:26.078835 24014 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:26.079073 23317 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:26.079958 23317 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:26.081260 23317 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:26.084344 24014 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:26.085482 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:26.086292 24686 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:26.086772 24686 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:26.087678 24686 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:26.089213 23317 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:26.098810 23214 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:26.314318 23466 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:26.316205 24628 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:26.337206 23316 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:26.345060 23390 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:26.346794 24628 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:26.349097 24526 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:26.383455 23466 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:26.385223 24602 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:26.656191 24014 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:26.657500 24649 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:26.669998 23316 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:26.676682 24710 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:26.820537 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:26.860533 23468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d137914eb14302bb888a191819fb75"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:26.861042 23468 raft_consensus.cc:606] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:26.863925 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:26.864392 23390 raft_consensus.cc:606] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:26.867007 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:26.867460 23390 raft_consensus.cc:606] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:26.869844 23317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:26.870324 23317 raft_consensus.cc:606] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:26.871568 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:13:26.873238 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:26.874476 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:26.983711 23466 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:26.990749 23390 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:26.990849 23436 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03644bf5f16347c98fd9217f8a7fab79. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:26.992157 23317 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:26.993019 24710 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:26.993888 24686 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:27.002489 24710 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.004637 23390 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.004140 23317 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.019546 23218 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.035301 23390 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:27.045696 23317 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:27.047087 24574 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:27.047991 23466 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:13:27.048053 23362 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b4ebaab4fff4457a89f5a00d4f71ddc8. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.051829 24574 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:27.055699 24628 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.057387 23317 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.067435 23466 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.077203 23214 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.079173 23317 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:27.085554 23287 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 709006b08c044ed38e2df81795ed1613. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:27.086601 23466 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:27.088070 23390 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:27.089350 24602 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:27.090381 24722 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:13:27.110857 24526 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.112536 23390 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.123646 23468 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:13:27.130750 23267 auto_rebalancer.cc:629] Failed to schedule move for tablet 06d137914eb14302bb888a191819fb75: Remote error: Service unavailable: leader transfer in progress
I20260709 12:13:27.136134 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:27.156662 24574 raft_consensus.cc:993] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:27.157152 24574 raft_consensus.cc:1081] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:27.160409 23317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:27.161136 23317 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:27.161511 23317 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.173767 23317 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.182022 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:27.182535 23390 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:27.182736 23317 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:27.182871 23390 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:27.183882 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:27.184629 23468 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.183894 23390 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.187274 23390 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.191396 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:27.192633 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:27.193411 24526 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:27.193892 24526 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:27.194717 24526 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.204242 23390 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:27.207772 23217 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.228969 24628 raft_consensus.cc:993] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:27.229502 24691 raft_consensus.cc:1081] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:27.231129 23468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:13:27.231771 23468 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:27.232122 23468 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.236814 23468 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.238673 23468 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:27.239213 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:27.239632 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:27.239864 23390 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:27.240144 24015 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.240168 23390 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:27.241135 23390 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.242347 23390 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.248721 23390 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:27.248715 24015 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:27.250422 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:27.251137 24710 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:27.251616 24710 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:27.252521 24710 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.262724 24649 raft_consensus.cc:993] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:27.263244 24686 raft_consensus.cc:1081] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:27.263906 23217 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.267908 23390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06d137914eb14302bb888a191819fb75"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:13:27.268500 23390 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:27.268877 23390 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.282248 23390 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.284159 23390 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:27.286496 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:27.287050 23466 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:27.287322 23466 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:27.288085 23466 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.289268 23466 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.290369 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:27.290954 23316 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:27.297155 23466 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:27.297155 23316 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:27.298465 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:27.299530 24691 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:27.300028 24691 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:27.300763 24691 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:27.311607 23217 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:27.345767 24722 raft_consensus.cc:993] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:27.346273 24722 raft_consensus.cc:1081] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:27.348053 24015 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca56a943ba904b779d7584f94c266344"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:27.348869 24015 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:27.349292 24015 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.357404 24015 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.359213 24015 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:27.359829 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:27.360426 23316 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:27.360535 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:27.360785 23316 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:27.361181 23390 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.361594 23316 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.362871 23316 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:27.368124 23390 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:27.369585 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:27.370522 24676 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:27.371019 24676 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:27.371872 24676 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:27.373468 23316 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:27.381989 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:27.545917 24727 ts_tablet_manager.cc:933] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:27.547850 24727 tablet_copy_client.cc:323] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:27.549746 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet 03644bf5f16347c98fd9217f8a7fab79 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:27.550206 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet 03644bf5f16347c98fd9217f8a7fab79 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-03644bf5f16347c98fd9217f8a7fab79
I20260709 12:13:27.556717 23480 tablet_copy_source_session.cc:215] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.559952 24727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03644bf5f16347c98fd9217f8a7fab79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.572372 24729 ts_tablet_manager.cc:933] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:27.577325 24727 tablet_copy_client.cc:806] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.578092 24727 tablet_copy_client.cc:670] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.579576 24729 tablet_copy_client.cc:323] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:27.582581 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 709006b08c044ed38e2df81795ed1613 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:27.582880 24727 tablet_copy_client.cc:538] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.583470 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 709006b08c044ed38e2df81795ed1613 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-709006b08c044ed38e2df81795ed1613
I20260709 12:13:27.591989 23331 tablet_copy_source_session.cc:215] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.594317 24727 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:27.595652 24729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709006b08c044ed38e2df81795ed1613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.612154 23468 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:27.616803 24526 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:27.625597 24727 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:27.626567 24727 tablet_bootstrap.cc:492] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:27.627265 24727 ts_tablet_manager.cc:1403] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.007s
I20260709 12:13:27.630487 24727 raft_consensus.cc:359] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:27.631310 24727 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:27.631991 24727 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:27.639367 24727 ts_tablet_manager.cc:1434] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:13:27.640964 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-03644bf5f16347c98fd9217f8a7fab79 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:27.641470 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-03644bf5f16347c98fd9217f8a7fab79 on tablet 03644bf5f16347c98fd9217f8a7fab79 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:27.646829 24727 ts_tablet_manager.cc:933] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:27.662395 24729 tablet_copy_client.cc:806] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.663663 24729 tablet_copy_client.cc:670] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.665606 24727 tablet_copy_client.cc:323] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:27.667788 24729 tablet_copy_client.cc:538] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.675603 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:27.676405 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet b4ebaab4fff4457a89f5a00d4f71ddc8 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-b4ebaab4fff4457a89f5a00d4f71ddc8
I20260709 12:13:27.730528 23316 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:27.732867 23466 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:27.736083 23406 tablet_copy_source_session.cc:215] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:27.736938 24628 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:27.740283 24574 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:27.749567 24727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ebaab4fff4457a89f5a00d4f71ddc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:27.762904 24727 tablet_copy_client.cc:806] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:27.763576 24727 tablet_copy_client.cc:670] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:27.769615 24729 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:27.773389 24727 tablet_copy_client.cc:538] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:27.799630 23390 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:27.813582 23390 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:27.828402 24649 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:27.863837 24727 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:27.871691 24015 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:27.875931 24649 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:27.917677 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:27.935838 23317 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:27.942736 23391 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:27.944767 24676 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:13:27.949602 24676 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:28.023751 24727 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:28.024745 24727 tablet_bootstrap.cc:492] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:28.025478 24727 ts_tablet_manager.cc:1403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.162s	user 0.025s	sys 0.011s
I20260709 12:13:28.028456 24727 raft_consensus.cc:359] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:28.029529 24727 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:28.030224 24727 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:28.042410 24727 ts_tablet_manager.cc:1434] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 12:13:28.044137 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:28.045220 24722 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.044816 23316 raft_consensus.cc:606] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:28.050112 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-b4ebaab4fff4457a89f5a00d4f71ddc8 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:28.050572 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-b4ebaab4fff4457a89f5a00d4f71ddc8 on tablet b4ebaab4fff4457a89f5a00d4f71ddc8 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:28.053858 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:28.054762 23466 raft_consensus.cc:606] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:28.059952 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:28.060621 23390 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:28.062304 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:28.069765 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:28.071758 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:28.079149 24729 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:28.080132 24729 tablet_bootstrap.cc:492] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:28.083947 24729 ts_tablet_manager.cc:1403] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.315s	user 0.081s	sys 0.106s
I20260709 12:13:28.089502 24729 raft_consensus.cc:359] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:28.090788 24015 raft_consensus.cc:1217] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:28.090682 24729 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:28.092905 24729 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:28.096184 24729 ts_tablet_manager.cc:1434] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:13:28.111099 24628 raft_consensus.cc:993] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:28.111541 24628 raft_consensus.cc:1081] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:28.114485 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-709006b08c044ed38e2df81795ed1613 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:28.114928 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-709006b08c044ed38e2df81795ed1613 on tablet 709006b08c044ed38e2df81795ed1613 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:28.115132 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:28.115674 23316 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:28.116017 23316 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.138588 23316 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.141842 23316 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:28.143011 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56803.7us
I20260709 12:13:28.144141 24649 raft_consensus.cc:993] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:28.150214 24649 raft_consensus.cc:1081] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:28.152531 23391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:13:28.153071 23391 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:28.153405 23391 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.156651 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:28.157210 23394 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:28.157505 23394 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:28.158241 23394 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.159427 23394 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.166220 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:28.168236 23391 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.171694 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:28.172190 24015 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.173770 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:28.175354 24198 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:28.175807 24198 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:28.176518 24198 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.183125 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:28.183827 23468 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:28.184147 23468 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:28.184841 23468 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.185643 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:28.186017 23468 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.186172 23470 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:28.192961 23468 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:28.195097 24014 raft_consensus.cc:1217] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:28.199797 23470 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:28.200309 24015 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:28.202574 23391 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:28.202648 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:28.204715 24574 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:28.205143 24574 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:28.205960 24574 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.235121 23217 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:28.256330 23214 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:28.309410 24526 raft_consensus.cc:993] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:28.309940 24751 raft_consensus.cc:1081] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:28.312124 23391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:28.312649 23391 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:28.312976 23391 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:28.318948 23391 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.320497 23391 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:28.321378 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:28.321533 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:28.322072 23320 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:28.322203 23468 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:28.322448 23320 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:28.323218 23320 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.324615 23320 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:28.329210 23468 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:28.330837 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:28.331825 24735 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:28.332154 23320 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:28.332293 24735 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:28.334096 24735 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:28.350569 23214 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:28.477326 24754 raft_consensus.cc:1064] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:28.480273 24754 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.486686 24015 raft_consensus.cc:1217] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:28.502527 24015 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.502939 23391 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.504194 24686 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.505604 24710 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.506958 23320 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:28.511494 24710 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.523382 24710 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.525568 23391 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.527321 23320 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.537575 24015 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.547365 23217 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.574563 23462 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.579788 24014 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:28.581296 23320 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:28.581321 24686 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:13:28.588544 24710 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.596737 24754 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.602721 24014 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.612068 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 03644bf5f16347c98fd9217f8a7fab79 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.616312 23462 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:28.617731 24198 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.618242 23218 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.621403 23316 raft_consensus.cc:2955] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.634776 24198 raft_consensus.cc:1064] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:28.638423 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 03644bf5f16347c98fd9217f8a7fab79 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:28.646391 24198 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.684043 24766 tablet_replica.cc:333] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:28.688190 24766 raft_consensus.cc:2243] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:28.690428 24766 raft_consensus.cc:2272] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:28.695560 24766 ts_tablet_manager.cc:1916] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:28.705698 23462 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.708390 23391 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:28.710759 23394 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.712818 24014 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.718108 24198 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.729668 24198 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.733551 24771 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:28.733870 24768 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.736363 24766 ts_tablet_manager.cc:1929] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:28.736876 24766 log.cc:1199] T 03644bf5f16347c98fd9217f8a7fab79 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/03644bf5f16347c98fd9217f8a7fab79
I20260709 12:13:28.738511 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 03644bf5f16347c98fd9217f8a7fab79 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:28.748931 24016 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.752584 24574 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.788508 24768 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.790457 23390 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.797076 23462 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:28.799981 24574 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.827131 24014 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.849633 23218 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.852101 23462 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:28.853343 24574 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:28.860252 23468 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.882917 23316 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.887128 24628 raft_consensus.cc:1064] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:28.892943 24628 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:28.908407 23394 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:28.910526 24014 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:28.911685 24768 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.914124 24768 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:28.920789 23316 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:28.926270 24574 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:28.922657 24768 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:28.952888 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48802.8us
I20260709 12:13:28.953287 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 709006b08c044ed38e2df81795ed1613 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:28.954748 24014 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.954991 23468 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.964740 24735 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.968972 24016 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.931070 23394 raft_consensus.cc:2955] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:28.974844 24735 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:28.984025 23320 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:28.989024 24628 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:13:29.007349 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 4 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.031162 24783 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.038160 23320 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.050160 23468 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.058128 24016 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.103063 23214 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.114300 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 709006b08c044ed38e2df81795ed1613 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:36032
I20260709 12:13:29.134886 23390 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.136023 24790 tablet_replica.cc:333] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:13:29.141139 24014 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:29.142813 24788 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:29.144021 23462 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:29.145944 24790 raft_consensus.cc:2243] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:29.146677 24790 raft_consensus.cc:2272] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:29.147079 24783 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:29.153657 24790 ts_tablet_manager.cc:1916] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:29.162312 24735 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.169205 24014 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.178263 23468 raft_consensus.cc:2955] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.186961 24790 ts_tablet_manager.cc:1929] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:29.187547 24790 log.cc:1199] T 709006b08c044ed38e2df81795ed1613 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/709006b08c044ed38e2df81795ed1613
I20260709 12:13:29.190618 23217 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:29.194581 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:29.200716 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 709006b08c044ed38e2df81795ed1613 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:29.206074 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet b4ebaab4fff4457a89f5a00d4f71ddc8 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:29.213817 24791 tablet_replica.cc:333] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:29.214965 24791 raft_consensus.cc:2243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:29.215848 24791 raft_consensus.cc:2272] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:29.219867 24791 ts_tablet_manager.cc:1916] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:29.232472 24791 ts_tablet_manager.cc:1929] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:29.232784 24791 log.cc:1199] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/b4ebaab4fff4457a89f5a00d4f71ddc8
I20260709 12:13:29.234364 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet b4ebaab4fff4457a89f5a00d4f71ddc8 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:29.245927 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:29.285570 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "446014759a9c4a0f9c99759951367083"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:29.286104 23394 raft_consensus.cc:606] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:29.290253 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:29.291170 23320 raft_consensus.cc:606] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:29.295732 23390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:29.296203 23390 raft_consensus.cc:606] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:29.298550 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:29.299100 23320 raft_consensus.cc:606] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:29.300794 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:13:29.302457 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:29.303040 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:29.422106 24735 raft_consensus.cc:993] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:29.422602 24628 raft_consensus.cc:1081] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:29.424026 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "446014759a9c4a0f9c99759951367083"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:29.424571 23316 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:29.424888 23316 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.429771 23316 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.431375 23316 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:29.432281 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:29.432283 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:29.432976 24014 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.433120 23390 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:29.433723 23390 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:29.434451 23390 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.435544 23390 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.439669 24014 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:29.440950 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:29.441077 23390 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:29.443573 24768 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:29.444119 24768 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:29.444932 24768 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.467057 23217 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.489517 24735 raft_consensus.cc:993] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:29.490042 24735 raft_consensus.cc:1081] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:29.491600 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:29.492048 23320 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:29.492376 23320 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.493213 24198 raft_consensus.cc:993] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:29.493773 24198 raft_consensus.cc:1081] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:29.496289 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:29.496735 23470 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:29.497040 23470 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.499130 23320 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.500752 23320 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:29.501591 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:29.501504 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:29.501838 23470 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.502171 23468 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.502211 23390 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:29.502632 23390 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:29.503350 23390 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.504343 23390 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.504714 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:29.505239 23394 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.503711 23470 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:29.508870 23468 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:29.510828 23390 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:29.511009 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:29.511297 23394 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:29.511950 24768 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:29.512418 24768 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:29.512531 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:29.513317 24768 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.514731 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:29.515290 23320 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:29.515573 23320 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:29.516463 24710 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:29.516322 23320 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.516924 24710 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:29.517639 23320 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.517800 24710 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:29.526266 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:29.529323 23216 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.531028 23320 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:29.566838 24768 raft_consensus.cc:993] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:29.567370 24198 raft_consensus.cc:1081] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:29.568867 24014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:29.569559 24014 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:29.569917 24014 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.575418 24014 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.577687 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:29.577778 24014 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:29.577953 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:29.578250 23320 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:29.578510 23394 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.578635 23320 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:29.579532 23320 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.580788 23320 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:29.585561 23394 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:29.586632 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:29.587312 24789 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:29.587692 24789 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:29.588503 24789 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:29.589627 23320 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:29.597756 23216 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:29.929418 23470 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:29.930840 24198 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:29.941244 23390 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:29.944298 24198 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:13:29.971187 23394 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:29.978672 24710 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:29.988317 23316 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:29.989816 24789 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:29.991721 23394 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:29.995031 24014 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:29.996423 24635 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:30.002990 24198 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:30.037547 23316 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:30.038458 23392 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:30.042495 24686 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:30.042860 24771 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:13:30.176856 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 130493us
W20260709 12:13:30.183741 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 135027us
W20260709 12:13:30.184908 24813 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:13:30.189335 24812 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:13:30.277031 23468 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:30.284833 23393 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:30.285239 23316 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:30.285392 23436 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b055a413d01c460895f3bc5034afc461. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:30.286245 24815 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:30.287065 24710 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:30.295030 24686 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.296579 23393 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.299208 23316 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.309612 23390 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:30.316262 23216 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.331418 23316 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:30.331604 23468 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:13:30.331532 23362 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff06250da5d340babc2a28fd336380a7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:30.333549 24821 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.334360 24628 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.341832 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:30.352761 24821 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.362773 23320 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.369136 23462 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.378690 23216 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:30.394686 23316 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:30.408406 23468 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:13:30.409010 23287 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3d1af6a7ce14f5387824508ce04bfd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:30.410132 23390 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:30.412550 24771 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.414610 24198 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:30.420256 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:30.421734 23316 raft_consensus.cc:606] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:30.424537 23462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:30.424525 24807 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.425691 23462 raft_consensus.cc:606] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:30.425974 23468 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.431006 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:30.431582 23316 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:30.432695 23390 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.434710 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:30.436543 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:30.437368 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:30.442754 23392 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:30.447701 24771 raft_consensus.cc:993] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:30.449540 24768 raft_consensus.cc:1081] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:30.451936 23393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:30.452534 23393 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:30.452989 23393 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:30.453586 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.468057 23393 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:30.484508 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:30.485131 23316 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:30.485419 23316 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:30.486255 23316 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:30.487340 23316 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:30.493949 23393 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:30.495165 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:30.495723 23468 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:30.496038 23316 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:30.498960 23462 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:13:30.499423 23362 consensus_peers.cc:597] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2349832eeb6f4824800ff39a557daeb6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:30.499485 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:30.500017 23320 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:30.500370 24628 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:30.500818 24628 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:30.501612 24628 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:30.503039 24821 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:30.505533 24735 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:30.515667 23468 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:30.662899 23462 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.878782 24832 ts_tablet_manager.cc:933] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:30.879580 24833 ts_tablet_manager.cc:933] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:30.894512 24828 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:30.924508 24832 tablet_copy_client.cc:323] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:30.926244 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet b055a413d01c460895f3bc5034afc461 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:30.926779 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet b055a413d01c460895f3bc5034afc461 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-b055a413d01c460895f3bc5034afc461
I20260709 12:13:30.945345 23480 tablet_copy_source_session.cc:215] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:30.978181 24833 tablet_copy_client.cc:323] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:31.103804 24198 raft_consensus.cc:993] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:31.104425 24198 raft_consensus.cc:1081] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:31.114962 23395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:31.115518 23395 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:31.115860 23395 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.126029 24843 ts_tablet_manager.cc:933] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:31.128509 24832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b055a413d01c460895f3bc5034afc461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:31.129696 23395 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:31.131731 23395 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:31.135344 24843 tablet_copy_client.cc:323] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:31.137409 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:31.137990 23319 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:31.138293 23319 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:31.139053 23319 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:31.140169 23319 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.144613 23320 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:31.147332 23319 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:31.150043 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:31.150959 24831 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:31.151417 24831 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:31.152225 24831 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:31.154017 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet ff06250da5d340babc2a28fd336380a7 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:31.154584 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet ff06250da5d340babc2a28fd336380a7 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-ff06250da5d340babc2a28fd336380a7
I20260709 12:13:31.152318 23216 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:31.170073 23406 tablet_copy_source_session.cc:215] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:31.170298 24832 tablet_copy_client.cc:806] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:31.174059 24832 tablet_copy_client.cc:670] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:31.171092 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:31.174686 24014 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.184029 24014 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:31.199812 24833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff06250da5d340babc2a28fd336380a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:31.211862 24832 tablet_copy_client.cc:538] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:31.216338 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet c3d1af6a7ce14f5387824508ce04bfd0 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:31.216992 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet c3d1af6a7ce14f5387824508ce04bfd0 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-c3d1af6a7ce14f5387824508ce04bfd0
W20260709 12:13:31.234138 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44379.9us
I20260709 12:13:31.243659 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:31.247465 24686 raft_consensus.cc:993] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:31.247922 24686 raft_consensus.cc:1081] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:31.274292 24832 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:31.275978 24855 ts_tablet_manager.cc:933] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:31.277245 23331 tablet_copy_source_session.cc:215] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:31.288223 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42792
I20260709 12:13:31.290023 23320 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:31.290529 23320 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.316887 24843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d1af6a7ce14f5387824508ce04bfd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:31.334089 23320 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:31.339797 24855 tablet_copy_client.cc:323] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:31.344602 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:31.345000 23320 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:31.345248 23387 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.346908 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 2349832eeb6f4824800ff39a557daeb6 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:31.347487 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 2349832eeb6f4824800ff39a557daeb6 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-2349832eeb6f4824800ff39a557daeb6
I20260709 12:13:31.348521 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:31.349086 23470 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:31.349355 23470 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:31.351845 23470 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:31.353044 23470 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:31.362895 23406 tablet_copy_source_session.cc:215] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:31.374274 23387 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:31.382133 24833 tablet_copy_client.cc:806] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:31.383162 24833 tablet_copy_client.cc:670] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:31.383798 24855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2349832eeb6f4824800ff39a557daeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:31.390774 24843 tablet_copy_client.cc:806] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:31.393185 24843 tablet_copy_client.cc:670] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:31.382220 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:31.397163 24807 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:31.397668 24807 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:31.398452 24807 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:31.403254 24833 tablet_copy_client.cc:538] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:31.407488 23462 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:31.409307 24628 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:31.411599 24843 tablet_copy_client.cc:538] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:31.433157 23214 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:31.440398 23217 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:31.442137 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:31.446417 23320 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:31.447865 24821 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:31.476132 24833 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:31.479190 24855 tablet_copy_client.cc:806] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:31.485153 24855 tablet_copy_client.cc:670] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:31.490733 24843 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:31.502312 24855 tablet_copy_client.cc:538] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:31.535578 24832 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:31.536569 24832 tablet_bootstrap.cc:492] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:31.537253 24832 ts_tablet_manager.cc:1403] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.263s	user 0.018s	sys 0.013s
I20260709 12:13:31.556816 24832 raft_consensus.cc:359] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.557716 24832 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:31.558331 24832 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.570516 24832 ts_tablet_manager.cc:1434] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:13:31.572274 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-b055a413d01c460895f3bc5034afc461 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:31.572742 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-b055a413d01c460895f3bc5034afc461 on tablet b055a413d01c460895f3bc5034afc461 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:31.585857 24855 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:31.593598 24833 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:31.594585 24833 tablet_bootstrap.cc:492] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:31.595263 24833 ts_tablet_manager.cc:1403] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.119s	user 0.020s	sys 0.005s
I20260709 12:13:31.597847 23320 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:31.599162 24628 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:31.611557 24833 raft_consensus.cc:359] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.612475 24833 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:31.613118 24833 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.642700 24011 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:31.652892 24628 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:31.663587 24833 ts_tablet_manager.cc:1434] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.068s	user 0.009s	sys 0.000s
I20260709 12:13:31.665730 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ff06250da5d340babc2a28fd336380a7 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:31.666174 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ff06250da5d340babc2a28fd336380a7 on tablet ff06250da5d340babc2a28fd336380a7 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:31.673537 24855 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:31.674487 24855 tablet_bootstrap.cc:492] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:31.675145 24855 ts_tablet_manager.cc:1403] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.099s	user 0.018s	sys 0.004s
I20260709 12:13:31.678066 24855 raft_consensus.cc:359] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.678855 24855 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:31.679486 24855 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.682129 24855 ts_tablet_manager.cc:1434] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:13:31.684345 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-2349832eeb6f4824800ff39a557daeb6 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:13:31.684747 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-2349832eeb6f4824800ff39a557daeb6 on tablet 2349832eeb6f4824800ff39a557daeb6 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:31.688588 24843 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:31.689564 24843 tablet_bootstrap.cc:492] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:31.690191 24843 ts_tablet_manager.cc:1403] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.200s	user 0.028s	sys 0.021s
I20260709 12:13:31.694437 24843 raft_consensus.cc:359] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.711611 24843 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:31.712168 24843 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:31.713928 24843 ts_tablet_manager.cc:1434] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:13:31.716060 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c3d1af6a7ce14f5387824508ce04bfd0 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:31.716599 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-c3d1af6a7ce14f5387824508ce04bfd0 on tablet c3d1af6a7ce14f5387824508ce04bfd0 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:31.751209 24015 raft_consensus.cc:1217] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:31.908284 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:31.928249 24011 raft_consensus.cc:1217] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:31.962893 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:31.963503 23315 raft_consensus.cc:606] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:31.966161 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:31.966627 23396 raft_consensus.cc:606] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:31.969130 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:31.969722 23396 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:31.971112 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:31.972711 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:13:31.972929 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002c36a0 after lost signal to thread 22110
I20260709 12:13:31.973237 23462 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:31.973521 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:31.974081 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003149c0 after lost signal to thread 23253
W20260709 12:13:31.975185 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5900 after lost signal to thread 23345
W20260709 12:13:31.976089 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080011e1c0 after lost signal to thread 23420
W20260709 12:13:31.977077 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800357c00 after lost signal to thread 23494
I20260709 12:13:31.978662 24869 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:13:31.986712 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800352940 after lost signal to thread 24040
I20260709 12:13:32.011844 23396 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:32.014361 24869 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:32.023872 24841 raft_consensus.cc:993] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:32.024364 24841 raft_consensus.cc:1081] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:32.029469 23463 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:32.124082 23463 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:32.124580 23463 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:32.151337 23463 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.192023 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:32.192734 23396 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:32.193295 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:32.196498 23320 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:32.196836 23320 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:32.198101 23320 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.199337 23320 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:32.208189 24011 raft_consensus.cc:1217] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:13:32.222407 23463 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:32.226866 23396 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:32.228124 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:32.228937 24851 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:32.229417 24851 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:32.230253 24851 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.231140 23320 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:32.244805 23217 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 4 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:32.277655 24016 raft_consensus.cc:1217] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:32.283324 24686 raft_consensus.cc:1064] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:32.285770 24686 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.298192 23396 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:32.298473 24016 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:32.299714 24844 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:32.309505 24851 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:32.332489 23320 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:32.335457 24851 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:32.339993 24821 raft_consensus.cc:993] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:32.340503 24821 raft_consensus.cc:1081] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:32.335662 24830 raft_consensus.cc:1064] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:32.347774 24830 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.350283 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:32.350885 23320 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:32.351220 23320 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.364002 24821 raft_consensus.cc:993] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:32.364529 24821 raft_consensus.cc:1081] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:32.370160 23319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:32.370735 23319 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:32.371130 23319 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.390916 23320 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.395596 23320 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:32.398384 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:32.398931 23470 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.410553 23470 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:32.392741 23319 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.411906 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:32.416123 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:32.416710 23387 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:32.416982 23387 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:32.415730 24841 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:32.420357 24841 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:32.421176 24841 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.426920 24851 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.431807 23387 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:32.433194 23387 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.440304 23320 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.442591 23387 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:32.442210 24011 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.443260 24831 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.482064 24016 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.483513 23470 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.491537 23319 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:32.491355 24831 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.498832 23396 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.507789 24830 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.499328 23216 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:32.530391 23219 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:32.537873 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:32.539237 24011 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.538668 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:32.541699 23387 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:32.543102 23387 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:32.555095 23387 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.572391 23387 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:32.582809 23316 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.612514 24011 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:32.618119 24831 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.627072 23387 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
W20260709 12:13:32.645663 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59170.5us
I20260709 12:13:32.654417 23463 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.655766 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:32.658383 24839 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:32.659135 24839 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:32.662266 23315 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.660005 24839 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.697208 24876 raft_consensus.cc:1064] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:32.687107 24015 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.699915 24876 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.703871 23470 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.718243 24011 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:32.718730 23216 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.720629 24686 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.719731 23315 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:32.730118 24686 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.737604 24686 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.743873 23315 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.742285 23396 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:32.748229 23470 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.748593 23387 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.748324 24016 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:13:32.752278 24868 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.753252 24869 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.759474 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b055a413d01c460895f3bc5034afc461 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:32.759729 24839 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.764242 24015 raft_consensus.cc:2955] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.769025 23217 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 4, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 1 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:32.775255 24014 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:32.775583 23463 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:32.777001 24889 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:32.779085 24890 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:32.782407 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet b055a413d01c460895f3bc5034afc461 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:32.804064 23396 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:32.807260 24844 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:32.796765 23216 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:32.842029 23320 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:32.840214 24895 tablet_replica.cc:333] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:32.845068 24895 raft_consensus.cc:2243] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:32.840826 24628 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:32.847184 24686 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:13:32.910742 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67372us
I20260709 12:13:32.896155 24895 raft_consensus.cc:2272] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:13:32.941581 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 101812us
W20260709 12:13:32.947546 24893 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.094s	user 0.007s	sys 0.011s
I20260709 12:13:32.987897 23470 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.006574 24895 ts_tablet_manager.cc:1916] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:33.012591 24821 raft_consensus.cc:1064] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:33.016027 24821 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:33.028091 24016 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.032698 24807 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.050709 23468 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:33.052397 23396 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:33.053345 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ff06250da5d340babc2a28fd336380a7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:33.067363 24011 raft_consensus.cc:2955] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.074925 24895 ts_tablet_manager.cc:1929] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:33.081863 23217 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 4 to 5, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:33.076318 24895 log.cc:1199] T b055a413d01c460895f3bc5034afc461 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/b055a413d01c460895f3bc5034afc461
I20260709 12:13:33.100394 23315 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:33.102552 23468 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:33.103698 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet b055a413d01c460895f3bc5034afc461 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:33.103667 24821 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:13:33.106158 24905 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:33.109723 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49321.9us
I20260709 12:13:33.112130 24768 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:33.116528 24015 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:33.115505 23468 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.127204 24015 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:33.133062 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet ff06250da5d340babc2a28fd336380a7 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:33.139647 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:33.142036 24897 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:33.147477 23218 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.143265 24876 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:13:33.176924 23396 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:33.180493 24912 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:33.205598 24839 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:33.207456 24915 tablet_replica.cc:333] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:33.209815 24915 raft_consensus.cc:2243] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:33.210623 24915 raft_consensus.cc:2272] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:33.214844 24830 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.225415 23463 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.227995 23315 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.233736 24915 ts_tablet_manager.cc:1916] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:33.260808 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:33.261920 23316 raft_consensus.cc:606] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:33.265216 23319 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:33.269493 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:33.270135 23320 raft_consensus.cc:606] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:33.274348 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:33.274834 23466 raft_consensus.cc:606] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:33.276887 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:33.278826 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:33.279608 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:33.290819 23396 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.294687 23463 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:33.295547 24014 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:33.345134 24014 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.370200 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.384279 24912 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:33.392901 23387 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:33.400815 23468 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:33.402695 24014 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:13:33.403209 24890 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:33.415742 24839 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:33.416571 24890 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:33.423985 24830 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.425980 24900 raft_consensus.cc:993] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:33.426460 24900 raft_consensus.cc:1081] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:33.436610 24016 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.444011 24868 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.447767 24807 raft_consensus.cc:993] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:33.448261 24912 raft_consensus.cc:1081] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:33.450078 23463 raft_consensus.cc:2955] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.454667 23468 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.471668 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c3d1af6a7ce14f5387824508ce04bfd0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:33.473268 23387 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:33.473938 23387 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:33.474339 23387 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.475740 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.478641 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42792
I20260709 12:13:33.479249 23315 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:33.479698 23315 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.484064 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2349832eeb6f4824800ff39a557daeb6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:13:33.497792 23218 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 3 to 4, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.499189 24016 raft_consensus.cc:2955] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:33.499377 23387 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.502893 23387 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:33.510113 24915 ts_tablet_manager.cc:1929] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:33.510597 24915 log.cc:1199] T ff06250da5d340babc2a28fd336380a7 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/ff06250da5d340babc2a28fd336380a7
I20260709 12:13:33.505113 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:33.511616 23319 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:33.512002 23319 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:33.512188 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:33.514268 23463 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.514746 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet c3d1af6a7ce14f5387824508ce04bfd0 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:52264
I20260709 12:13:33.515642 24895 tablet_replica.cc:333] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:13:33.512949 23319 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.516636 24895 raft_consensus.cc:2243] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:33.516897 23319 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.517442 24895 raft_consensus.cc:2272] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:33.524762 24895 ts_tablet_manager.cc:1916] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:33.531247 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet ff06250da5d340babc2a28fd336380a7 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:33.532694 24868 raft_consensus.cc:993] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:33.533272 24912 raft_consensus.cc:1081] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:33.534973 23396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:33.535526 23396 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:33.535859 23396 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.545938 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 2349832eeb6f4824800ff39a557daeb6 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42786
I20260709 12:13:33.549870 24915 tablet_replica.cc:333] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:13:33.543608 23396 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.552160 24915 raft_consensus.cc:2243] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:13:33.553146 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:33.553784 23320 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:33.554075 23320 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:33.554874 23463 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:33.554775 23320 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.555886 23320 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.556962 24915 raft_consensus.cc:2272] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:13:33.558557 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:33.560125 24895 ts_tablet_manager.cc:1929] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:13:33.560290 24890 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:33.560609 24895 log.cc:1199] T c3d1af6a7ce14f5387824508ce04bfd0 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/c3d1af6a7ce14f5387824508ce04bfd0
I20260709 12:13:33.560925 24890 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:33.561888 24890 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.565200 24915 ts_tablet_manager.cc:1916] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:13:33.565658 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet c3d1af6a7ce14f5387824508ce04bfd0 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:33.572417 23320 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:33.573824 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:33.575281 24897 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:33.575753 24897 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:33.576615 24897 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.577885 23319 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:33.586577 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:33.587088 23463 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.589675 23396 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:33.592072 23315 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.592842 23218 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:33.594261 23315 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:33.594887 23218 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:33.598152 23463 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:33.601235 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:33.601804 23396 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.606060 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:33.606649 23463 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:33.606945 23463 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:33.610289 23396 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:33.612226 23463 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.613080 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:33.613786 24868 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:33.614266 24868 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:33.614802 23463 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:33.615016 24868 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:33.615688 24915 ts_tablet_manager.cc:1929] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:13:33.616115 24915 log.cc:1199] T 2349832eeb6f4824800ff39a557daeb6 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/2349832eeb6f4824800ff39a557daeb6
I20260709 12:13:33.621173 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 2349832eeb6f4824800ff39a557daeb6 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:13:33.622394 23463 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:33.633731 23218 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:33.944626 23463 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:33.945812 24897 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:33.973915 23320 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:33.977013 24830 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:33.988435 23319 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:33.989882 24830 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:34.014537 23468 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:34.030946 24905 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:34.120031 23463 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:34.121565 24868 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:34.139531 23396 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:34.143738 24912 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:34.373822 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:34.427827 23319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:34.428503 23319 raft_consensus.cc:606] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:34.431394 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:34.431952 23396 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:34.436074 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:34.436671 23396 raft_consensus.cc:606] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:34.438323 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:34.440061 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:34.440831 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:34.568408 24890 raft_consensus.cc:993] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:34.568931 24890 raft_consensus.cc:1081] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:13:34.585389 23319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:13:34.586046 23319 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:34.586434 23319 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:34.598946 23319 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:34.600853 23319 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:34.623751 24912 raft_consensus.cc:993] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:34.624233 24912 raft_consensus.cc:1081] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:34.626868 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:34.627497 23392 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:34.627812 23392 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:34.628594 23392 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:34.629935 23392 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:34.651012 23396 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:34.664535 23392 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:34.665939 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:34.666782 24839 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:34.667306 24839 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:34.668164 24839 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:34.670945 23463 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:34.671555 23463 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:34.671917 23463 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:34.675470 23319 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:34.676517 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:34.676784 23470 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:13:34.677044 23468 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:34.679171 24830 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:34.705919 24905 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:35.516391 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:35.743711 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080019bac0 after lost signal to thread 22110
I20260709 12:13:35.766248 24940 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.767681 24940 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.771749 24940 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:35.790594 23468 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.113s	user 0.004s	sys 0.019s
I20260709 12:13:35.791170 23468 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:35.792029 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2019) took 1123 ms. Trace:
I20260709 12:13:35.792526 23468 rpcz_store.cc:276] 0709 12:13:34.668971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.676284 (+  7313us) service_pool.cc:224] Handling call
0709 12:13:35.791836 (+1115552us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816}
I20260709 12:13:35.832500 24944 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.833966 24944 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.834399 24940 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.835949 24940 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.837188 24944 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.839069 24936 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:35.839422 24940 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:35.840054 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080024a720 after lost signal to thread 23253
W20260709 12:13:35.840466 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800261ec0 after lost signal to thread 23267
I20260709 12:13:35.839903 24936 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.856488 24947 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.854081 24944 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.858914 24947 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.841349 24946 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:35.859067 24944 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.862941 24947 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.862946 24944 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.843451 24936 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:35.867415 24951 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.868925 24946 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.879171 24946 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.882556 24851 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:35.874599 24951 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.883388 24851 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.891409 24851 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:35.894002 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1401) took 1211 ms. Trace:
I20260709 12:13:35.907337 24899 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:35.894634 23319 rpcz_store.cc:276] 0709 12:13:34.681922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.841011 (+1159089us) service_pool.cc:224] Handling call
0709 12:13:35.842016 (+  1005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.842762 (+   746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.842805 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.844155 (+  1350us) log.cc:844] Serialized 513 byte log entry
0709 12:13:35.892321 (+ 48166us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:35.892348 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.893328 (+   980us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:35.893421 (+    93us) raft_consensus.cc:1685] finished
0709 12:13:35.893464 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.893695 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":47765,"threads_started":1,"wal-append.queue_time_us":40714}
I20260709 12:13:35.909198 24944 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.918002 24948 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.913726 24899 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.934311 24954 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:35.934585 24943 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.935557 24943 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.938234 24943 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:35.938479 23463 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.266s	user 0.026s	sys 0.001s
I20260709 12:13:35.939162 24954 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.914358 24944 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.946525 24941 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.947469 24941 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.938930 23463 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.949587 24947 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.950145 24944 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.950465 24947 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.952515 24955 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:35.953887 24955 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.932202 24936 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:35.925628 24940 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.970083 24940 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.938658 24950 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.956760 24936 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:35.963727 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.281s	user 0.005s	sys 0.007s
I20260709 12:13:35.963631 24942 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:35.964214 24899 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.972604 24955 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:35.957230 24851 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:35.953208 24947 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:35.981622 23463 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:35.925275 24948 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.981274 24851 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.972666 24940 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:35.984613 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36042 (request call id 2020) took 1315 ms. Trace:
I20260709 12:13:35.969324 24943 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:35.985512 23463 rpcz_store.cc:276] 0709 12:13:34.669367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.670822 (+  1455us) service_pool.cc:224] Handling call
0709 12:13:35.983668 (+1312846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:35.975934 24944 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:35.990316 24950 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:35.992844 24953 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:35.995595 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.312s	user 0.005s	sys 0.009s
I20260709 12:13:35.972858 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:35.997637 24936 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:35.999547 24943 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:35.994112 24950 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:35.993773 24953 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.010756 24912 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.015714 24944 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.016389 24948 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.004449 24936 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:35.994616 24942 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.011797 24941 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.019039 24944 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:36.007895 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 1.32521e+06us
I20260709 12:13:36.035759 24940 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.019330 24851 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.041203 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.047135 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.047340 24960 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.047376 24954 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.050392 24957 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.011569 24912 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.070158 24953 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.069696 24957 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.071713 24955 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.072417 24955 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.075315 24957 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.075685 24955 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.049297 24948 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.076403 24946 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.076522 24948 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.077080 24946 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.042236 24941 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.078778 24948 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.079165 24946 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.079732 24946 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.080441 24948 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.080331 24946 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.081506 24956 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.082144 24956 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.086306 24956 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.043274 24944 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.042057 24936 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.046607 24851 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.091589 24955 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.091883 24964 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.092285 24955 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.092680 24964 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.094465 24946 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.094506 24955 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:36.052146 23362 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22fc2096e78f47158bbb4bae3a474e2c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:36.081149 24948 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.107280 24942 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.107532 24969 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.079195 24941 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.050988 24940 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:36.105717 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53928.4us
I20260709 12:13:36.108188 24969 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.111994 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.048131 24960 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.115072 24851 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.107285 24925 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.114903 24011 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:36.114303 24839 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.106302 24943 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:36.131585 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800036f40 after lost signal to thread 23345
I20260709 12:13:36.137591 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.125031 24851 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.121466 24957 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.110589 24953 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.137750 24966 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.140215 24839 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.121701 24899 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.134950 24951 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.125680 24955 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.131239 24964 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.121026 24969 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.165077 24851 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.163584 24943 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.125555 24942 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.121598 24963 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.131448 24944 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.189064 24948 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:36.202772 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800064c40 after lost signal to thread 23420
I20260709 12:13:36.192095 24951 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.190446 24946 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.206926 24946 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.208237 24941 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.205376 24974 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.204777 24943 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.204098 24950 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.208799 24941 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.207320 24830 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.195959 24899 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.216624 24973 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:36.216473 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08003358e0 after lost signal to thread 23494
I20260709 12:13:36.216384 24851 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.217304 24961 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.217892 24876 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.200655 24969 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.218389 24936 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.218804 24944 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.209760 24958 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.216708 24951 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:36.228827 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080035afe0 after lost signal to thread 24040
I20260709 12:13:36.209393 24974 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.232748 24830 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.235136 24839 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.235813 24876 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.261549 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.262218 24876 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.262331 23319 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
W20260709 12:13:36.260860 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 165378us
I20260709 12:13:36.264406 24951 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.236012 24950 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.201645 24955 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.238415 24899 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.196684 24953 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.123015 24925 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.194936 24957 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.209965 24942 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.198182 24966 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.258795 24940 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:36.246901 24941 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.259909 24830 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.237918 24946 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.256896 24905 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.252655 24948 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.304117 24948 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.285220 24974 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.285980 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.288178 24953 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.289394 24968 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.266441 24936 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.289569 24839 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.313694 24974 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.290119 24851 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.290239 24980 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.275848 24969 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.291186 24961 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.295440 24955 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.297001 24943 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.265166 24958 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.321133 24942 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.307850 24972 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.305801 24905 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.309640 24957 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.310956 23320 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:36.246068 24963 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.313911 24839 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.320851 24984 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.313637 24936 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.321409 24925 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.322590 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.313154 24975 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.315171 24978 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.334336 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.335624 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.313611 24968 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.277098 24973 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.332015 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.337934 24830 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.341511 24971 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.341745 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.342155 24011 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:36.342363 24966 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.342850 24905 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.343016 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.343438 24936 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.343847 24839 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.343935 24982 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.318182 24946 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.319495 24956 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.325598 24974 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.353223 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.344732 24960 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.347468 24944 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.344280 24972 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.356551 24830 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.358086 23320 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:36.350610 24953 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:36.362026 24967 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.362387 24953 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.362475 24941 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.361590 24965 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.361706 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.359745 24969 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.364873 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.362888 24953 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:36.365824 23315 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:36.367941 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.351615 24981 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.362681 24967 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.367451 24962 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.359579 24890 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.354568 24958 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.370398 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.365049 24971 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.360666 24966 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.352284 24979 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.373744 24962 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.363471 24905 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.353564 24984 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.380537 24905 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.382197 24984 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.375197 24946 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.384234 24946 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.382458 24962 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.363159 24944 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.387079 24944 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.375864 24963 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.375306 24973 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.374639 23980 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.394248 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.394408 23462 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:36.394870 23469 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
W20260709 12:13:36.396488 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76413.6us
I20260709 12:13:36.393230 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.378793 24958 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:36.398754 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47535.9us
I20260709 12:13:36.365319 24975 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.387653 24890 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.352947 24980 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.390542 24961 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.392586 24968 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.379187 24983 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.409858 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.408921 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.410471 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.411721 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.413026 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.398872 24969 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.414018 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.413491 23218 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:36.408939 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.378854 24956 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.400609 24905 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.416534 24013 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:36.380541 24979 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.391904 24965 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.436252 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.420408 24969 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.414835 24966 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.446003 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.363497 24978 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.420699 24958 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.405560 24981 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.434356 24983 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.395180 24982 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.475755 24830 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.444190 24969 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.482646 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.440795 24936 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.449977 24912 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.488616 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.485492 24925 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:36.494529 24925 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.494923 24925 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.495412 24966 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.496028 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.500293 23387 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.504129 24963 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.478330 24974 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.503588 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.519306 24985 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:36.520013 24985 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.520519 24985 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:36.522245 24967 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.524775 24971 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.525956 24899 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.526096 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.526680 24899 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.527339 24966 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.527973 24966 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.504323 24957 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:36.502242 24950 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.506110 24963 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.523262 24956 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.504290 24936 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.529632 24957 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.547572 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.545269 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.547005 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.559407 24013 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.532455 24972 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.571847 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.572593 24015 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.552317 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.577216 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.560354 24967 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.581310 24967 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.551430 24979 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.550119 24890 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.604342 24936 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.550622 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.605487 23317 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:36.539388 24974 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:36.606973 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.608214 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.608402 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.609839 24966 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.552099 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.612980 23980 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.615761 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.616130 24899 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.616956 24955 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:36.617365 24955 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.617753 24955 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.618595 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.606314 24963 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.622318 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.611107 24974 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.624993 24983 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.627029 24967 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.628844 24957 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.629663 24958 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.630376 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.631412 24989 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.632189 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.636070 24958 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.636554 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.636626 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.636957 24958 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.638928 24958 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.639770 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.641868 24982 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.641993 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.642254 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.642670 23320 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:36.642916 24978 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.643013 23320 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:36.643512 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.644224 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.632354 24974 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.623847 24948 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.643977 23320 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.647568 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.648924 23315 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.650484 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.650717 24982 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:36.644083 24965 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.651789 23320 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.650070 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.651144 24980 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.655232 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.655656 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.656857 23469 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.659453 24982 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.663137 24990 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.663187 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.664512 24981 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.664819 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.664758 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.670811 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.671090 24982 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.669736 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.671669 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.672955 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.673321 24936 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.675657 24936 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.675799 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.676219 24936 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.675686 24986 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.677210 24986 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.677323 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.675750 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.677639 24986 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.680758 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.684401 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.681344 23387 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:36.685775 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.631990 24989 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.686414 23468 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:36.687230 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.687302 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.664542 24990 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:36.692032 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.692090 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.692553 24989 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.665023 24975 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.672890 24925 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.697006 23320 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.697701 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.691865 24953 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.698247 23387 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.698912 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.664079 24985 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.696971 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.701956 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.702675 23316 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:36.702701 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.703589 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.704077 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.705981 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.706967 23392 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.704391 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.707820 24953 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.709468 24009 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:36.711427 24925 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.712817 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.713399 23392 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:13:36.714758 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.715553 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.716058 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.717072 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.717669 23387 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:36.718238 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.718834 23317 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:36.723762 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.723984 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.724123 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.724414 23395 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:36.723248 24936 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.725881 23470 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:36.722409 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.730005 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:36.730516 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.731069 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.731823 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.732589 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.729926 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.691303 24955 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.713114 24986 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.735857 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.740844 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.734578 24936 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.741187 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.740300 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.734834 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.741506 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.744624 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.744930 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.731456 24969 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.734649 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.739665 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.746697 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.751983 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.738850 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.747325 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.747498 24990 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.748229 23321 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.753080 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.729020 24974 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.754333 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.755537 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.739008 24985 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:36.758322 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.758450 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.759683 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.760278 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.760512 24955 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.761708 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.761999 24008 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:36.766036 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.766144 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.764381 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.768527 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.768781 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:36.769192 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51890.9us
I20260709 12:13:36.769295 24985 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:36.769685 24985 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.769865 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.770121 24985 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.770354 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.770653 24986 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:36.770968 23392 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.771651 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.772735 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.774040 24941 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.774721 24941 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.775310 24941 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.778436 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:36.779546 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.779608 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.781191 24981 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:36.781579 24936 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:36.781389 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:36.782320 24015 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.781381 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:36.782629 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:36.783215 24014 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.785702 24009 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:36.785004 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:36.782532 24936 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:36.782358 24981 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:36.787824 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.789165 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.789788 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:36.791323 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.792536 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.795171 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.795735 23469 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.795588 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.790845 24941 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.798178 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.799108 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.798338 23980 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.799523 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:36.794859 24969 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:36.788529 24936 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:36.789420 24981 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:36.799352 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.805825 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.801962 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.802171 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.807340 23462 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:36.801394 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.808269 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.808454 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.808867 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.791138 24953 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
W20260709 12:13:36.809312 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:36.809881 24899 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:36.810294 24899 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:36.811276 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.811172 24899 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.811811 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:36.811996 24936 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.812544 24899 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.813225 24936 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:36.813411 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:36.815534 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.821640 24981 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.824532 24014 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:36.825675 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.827073 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.830199 24981 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.830646 24925 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.831372 24009 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:36.833479 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:36.834244 23395 raft_consensus.cc:606] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.835458 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.836301 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.837541 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.839454 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.839726 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.805979 24941 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.841439 24015 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:36.841706 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.842083 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.842383 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.845055 24936 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:36.845870 24936 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:36.845361 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.846817 24936 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.845248 24953 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:36.847956 24953 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:36.848960 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.849753 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.851557 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.852608 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.852608 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.826753 24974 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:36.854440 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.856655 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.853183 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.859336 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.861411 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.863368 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:36.864480 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.864933 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.861573 24967 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.868191 24988 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.868722 24014 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:36.870113 23468 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:36.882043 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.886449 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.887156 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.887743 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.889519 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.895206 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.897217 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.849057 24008 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:36.899695 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.900228 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:36.901592 23392 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:36.903083 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.900890 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.903306 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.904284 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.906350 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:36.908900 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.909214 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:36.892463 24944 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.898820 24972 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:36.914785 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.936954 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.907152 24968 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.941999 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.943014 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.944054 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.938591 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.945585 24974 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:36.938666 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.934571 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:36.916188 24942 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:36.950891 24961 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.952545 24965 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:36.953399 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:36.848973 24953 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:36.954764 24957 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:36.915805 24936 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:36.949798 23463 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:36.957257 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:36.955040 24942 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:36.962998 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:13:36.916653 24988 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.974022 23392 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:36.932047 24944 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:36.937608 24972 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:37.020924 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.024240 23391 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:37.003618 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.024804 23391 raft_consensus.cc:614] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:37.025385 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74959.5us
I20260709 12:13:36.974502 24961 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:37.028178 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.028589 24961 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.028991 24969 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.029412 24969 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.025861 24972 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.029947 24969 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.034111 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:36.916497 24975 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:37.029395 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.020630 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.020264 24985 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.040719 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.051321 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.018909 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:37.044942 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.046177 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.025624 24944 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.049865 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:36.973845 24965 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:37.050777 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:37.057452 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 126266us
I20260709 12:13:37.057888 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.034149 24955 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.058883 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:37.059928 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 156683us
I20260709 12:13:37.062737 24985 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.051342 24942 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:37.065207 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.057879 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:37.054656 24971 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.050871 24941 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.067705 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.068233 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.060039 24965 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.061180 24925 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:36.973590 24983 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:13:37.079923 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.061758 24955 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.083302 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.061347 24975 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:37.085167 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.067935 24991 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.074719 24956 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.086140 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.088207 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.088824 24985 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.095543 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.095752 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.096997 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.096887 24995 ts_tablet_manager.cc:933] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:37.086434 24955 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.085381 24974 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.104458 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.104645 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.091382 25000 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.106040 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.094501 24988 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.100822 24980 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.099139 24971 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.115861 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.117466 24983 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:37.118798 24966 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.119129 24966 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.120097 24969 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.119951 24966 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.122123 24969 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.123111 24966 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.126444 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.140781 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:37.153828 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.153504 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:37.160229 24975 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.161311 24890 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.161764 24890 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.162586 24890 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.163172 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.163774 24890 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.164336 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.164759 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.165546 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.167544 24963 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:37.168074 24963 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:37.168493 24963 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.167485 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.169797 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.169893 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.170786 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:37.171476 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 86432.3us
I20260709 12:13:37.172446 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.172202 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.174803 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:37.175153 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.176064 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.169134 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.175379 24998 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.159114 24968 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.182616 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.184413 24995 tablet_copy_client.cc:323] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:13:37.185580 24998 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.186015 24998 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:37.186409 24998 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.186317 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.175571 24912 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.191576 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.192955 24010 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:37.198695 24965 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.199916 24944 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.179579 24997 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.200918 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:13:37.201574 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.203754 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.204845 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:37.205181 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.205744 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.206357 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.196913 24942 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.214468 23462 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:37.215693 23318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:37.190395 24912 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:37.218149 23318 raft_consensus.cc:606] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.203655 24978 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.202638 24969 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.200883 25004 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.210228 24990 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:37.224341 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.225274 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.226635 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.232720 24944 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:37.205384 24968 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.210187 24950 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.218564 25002 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.223316 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.210479 24948 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.232074 24965 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:37.217059 24974 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.240206 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:37.237026 24950 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:37.217211 24971 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:13:37.238123 24990 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.243306 24942 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.220929 24912 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.243845 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 22fc2096e78f47158bbb4bae3a474e2c from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:13:37.244246 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.250422 24982 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:37.251904 24998 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.255535 24955 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.257508 24972 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.262603 24950 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.267617 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:13:37.267761 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 22fc2096e78f47158bbb4bae3a474e2c from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-22fc2096e78f47158bbb4bae3a474e2c
I20260709 12:13:37.265250 24990 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.279223 24961 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.286214 23321 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.288900 24974 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:13:37.281536 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91860.2us
I20260709 12:13:37.290418 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.291971 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.289264 24990 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.294373 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.295240 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.295433 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.296005 25002 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.296396 25002 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:37.296833 24965 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.296984 24991 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.282873 24966 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.294961 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:13:37.300695 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 86550.8us
I20260709 12:13:37.300712 24958 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:37.302026 24958 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.303200 24971 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.302920 24958 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.304144 24958 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:37.305183 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.305871 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.306548 25003 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.306370 25002 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.308601 25002 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.307642 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.308597 24008 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.311487 24991 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.311661 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.309686 24971 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.313283 25003 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.311385 23464 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:37.316324 25003 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:37.315052 24961 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:37.318543 24955 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.320770 24998 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.319761 25003 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.314638 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.324674 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.331076 23469 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.333489 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.334730 25007 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:37.334847 24948 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.335240 25007 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.335304 24948 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.335837 25007 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:37.335893 24948 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.335197 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.337826 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.340057 24961 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.340478 24961 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.341871 24961 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.320861 24963 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.340615 24008 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:37.344273 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.342311 24968 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:37.346285 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.346377 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.347795 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.342310 24966 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.352272 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.351069 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.348170 24971 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.347054 24968 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.349224 24965 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.357995 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.359396 24971 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.358542 24968 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:37.351490 24941 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.350545 24985 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.353701 24998 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:37.364336 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.360690 24987 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.365593 24971 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.372225 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.374567 25008 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.373894 24980 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.364223 24972 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:37.376981 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:37.365859 24957 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.391490 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.393618 24963 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.396162 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.392032 24956 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.402704 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.401105 24966 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.408601 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.409160 23469 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.380409 24987 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:13:37.374639 24998 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:37.401476 24989 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.418112 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.431399 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.434949 24953 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.436568 24965 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:37.436940 24965 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.407099 24956 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.441049 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.438444 24999 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.484129 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.502736 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.503925 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.504953 24986 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.505085 25000 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.505743 24986 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.497576 23218 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 319) took 1222 ms. Trace:
I20260709 12:13:37.508074 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.437863 24965 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.515617 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.516600 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.514022 23469 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:37.512287 24953 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:37.518359 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 167759us
I20260709 12:13:37.518833 24988 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.521184 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.442909 23216 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.520896 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:37.523010 25010 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 12:13:37.524062 24998 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.498504 25007 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.526373 25007 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.526254 24950 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.531598 24965 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:37.520035 24899 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.535218 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.536214 23470 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.536720 25003 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.538199 23465 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:37.540233 25003 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:37.533576 24942 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.549522 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.550146 23216 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:37.552402 24944 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.550976 24967 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.554579 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.504951 24961 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.507238 23218 rpcz_store.cc:276] 0709 12:13:36.264317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.264624 (+   307us) service_pool.cc:224] Handling call
0709 12:13:37.486052 (+1221428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:36.608834 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:36.609633 (+   799us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:36.609680 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:13:36.609709 (+    29us) tablet.cc:662] Decoding operations
0709 12:13:36.610107 (+   398us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:36.610197 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:36.610230 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:36.649705 (+ 39475us) tablet.cc:701] Row locks acquired
0709 12:13:36.658000 (+  8295us) write_op.cc:260] PREPARE: finished
0709 12:13:36.667065 (+  9065us) write_op.cc:270] Start()
0709 12:13:36.693609 (+ 26544us) write_op.cc:276] Timestamp: P: 1783599216666993 usec, L: 0
0709 12:13:36.701569 (+  7960us) op_driver.cc:348] REPLICATION: starting
0709 12:13:36.733127 (+ 31558us) log.cc:844] Serialized 544 byte log entry
0709 12:13:37.201249 (+468122us) op_driver.cc:464] REPLICATION: finished
0709 12:13:37.369043 (+167794us) write_op.cc:301] APPLY: starting
0709 12:13:37.369296 (+   253us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:37.369448 (+   152us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:37.369597 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:37.385048 (+ 15451us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:37.387484 (+  2436us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:37.387566 (+    82us) write_op.cc:312] APPLY: finished
0709 12:13:37.387915 (+   349us) log.cc:844] Serialized 23 byte log entry
0709 12:13:37.405600 (+ 17685us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:37.407801 (+  2201us) write_op.cc:454] Released schema lock
0709 12:13:37.419726 (+ 11925us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":149433,"apply.run_cpu_time_us":2951,"apply.run_wall_time_us":125151,"num_ops":1,"prepare.queue_time_us":130490,"prepare.run_cpu_time_us":6276,"prepare.run_wall_time_us":172483,"raft.queue_time_us":204597,"raft.run_cpu_time_us":25972,"raft.run_wall_time_us":191666,"replication_time_us":477392,"thread_start_us":315052,"threads_started":4,"wal-append.queue_time_us":73155}]]}
I20260709 12:13:37.528479 24950 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.663465 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.665616 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:37.665822 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:37.665907 24948 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.667361 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:37.667680 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:37.668022 24948 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.669660 23318 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:37.673300 23389 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:37.677541 24961 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.675408 23388 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:37.692195 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:37.695708 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 343459us
I20260709 12:13:37.697027 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.696581 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.699221 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.673000 24912 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.704480 24999 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.705276 24961 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:37.705843 24961 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:37.706044 24912 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.706372 24961 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.702932 25017 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.709244 23321 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.703330 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.700220 24979 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.716995 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.716952 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.719835 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.721365 24999 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.715591 25000 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.690678 24971 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.728753 24971 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.729082 24010 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:37.734766 24968 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.735711 24961 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.737944 25017 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.739943 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.744793 24979 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.745146 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.747115 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.748203 25016 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.750891 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.751422 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.751958 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.753077 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:37.757313 23316 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.753130 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.756248 24899 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:37.758471 24941 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.756333 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:37.759394 24987 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.761358 24971 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.761545 24958 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.757741 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.762195 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.760145 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.763615 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.765008 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:37.769680 24950 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.769752 24987 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.771054 24944 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:37.763182 24965 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.772902 24944 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:37.769773 24899 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.774029 24957 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.759689 24979 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.774448 24957 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.777736 24957 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.779186 24957 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.780021 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:37.780807 24008 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:37.781816 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.782990 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:37.784010 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:37.786454 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.787423 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.790239 24958 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:37.791482 24956 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.791863 24956 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.792542 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.794171 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:37.794188 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.792752 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.795974 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.797246 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:37.792797 24956 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.798502 24982 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:34538 (request call id 12) took 3150 ms. Trace:
I20260709 12:13:37.798851 24956 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.799171 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.801990 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.803861 23389 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:37.736888 24968 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.806288 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:37.807512 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.808710 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.810746 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.811678 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.811890 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:37.813603 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.799944 24991 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:13:37.814898 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.815352 24991 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:37.815747 24991 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.816419 24009 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.817732 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.818380 23392 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.798919 24982 rpcz_store.cc:276] 0709 12:13:34.648317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.648460 (+   143us) service_pool.cc:224] Handling call
0709 12:13:34.664692 (+ 16232us) log.cc:844] Serialized 513 byte log entry
0709 12:13:37.798375 (+3133683us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":184,"raft.queue_time_us":1099,"raft.run_cpu_time_us":1590,"raft.run_wall_time_us":1590,"thread_start_us":6627,"threads_started":1,"wal-append.queue_time_us":7040,"wal-append.run_cpu_time_us":1924,"wal-append.run_wall_time_us":1001895}
I20260709 12:13:37.811964 24980 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:37.822641 24980 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.823558 24980 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.825049 23391 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:37.827185 24009 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:37.827658 24016 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:37.829036 23392 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:37.805584 24958 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:13:37.830273 24965 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.830668 24965 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:37.831585 24965 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:37.816661 24991 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.833150 24965 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.833500 24991 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:37.835749 23464 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:37.838851 24010 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:37.840977 24961 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:37.841904 24979 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:37.843029 24967 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.843386 24967 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:37.844241 24967 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.845907 24967 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.849567 24899 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:37.853520 24999 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:37.853793 25000 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:37.854249 25000 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:37.854528 24999 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:37.854902 24999 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:37.855136 25000 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.855770 24999 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.859164 25000 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:37.859551 24999 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:37.859493 24899 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:37.857725 23389 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:37.861122 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:37.862022 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.863021 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.863615 25017 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.864490 24972 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:37.864634 24944 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:13:37.866001 24950 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:37.873517 24950 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:37.874461 24950 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:37.880553 24975 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:37.880615 24972 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:37.883451 24912 raft_consensus.cc:993] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:37.883915 25017 raft_consensus.cc:1081] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:13:37.886806 23395 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:37.887436 23318 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.888981 23312 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:37.894131 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.899147 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:37.916409 23388 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:13:37.936760 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57752.4us
I20260709 12:13:37.938297 25000 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:37.939076 23316 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:37.942483 23312 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:37.943835 23406 tablet_copy_source_session.cc:215] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:37.976696 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:37.972949 24975 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:13:37.983284 23468 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:37.977339 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.986359 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.993499 23316 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:37.994989 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.004536 25016 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:38.006031 23313 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:38.000658 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:13:38.007537 23466 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:38.007925 23466 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:38.016425 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:37.990885 24991 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.047740 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:38.000283 24961 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:38.013343 24965 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 12:13:38.061688 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61608.1us
I20260709 12:13:38.063169 23466 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:38.066095 23436 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec4e3b5eed804f2a8c2a01e5d1ae5f53. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:38.067979 24941 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:13:38.071111 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:38.080224 23394 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:13:38.061695 25008 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.103382 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:38.105032 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46344.4us
I20260709 12:13:38.100922 23466 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:38.076615 23218 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 2, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.111207 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:38.134744 24995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22fc2096e78f47158bbb4bae3a474e2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:38.104771 24950 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.115617 24944 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.107667 24967 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
W20260709 12:13:38.171190 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 102035us
W20260709 12:13:38.171567 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60515.7us
I20260709 12:13:38.175336 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.177711 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:38.178321 23389 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:38.182390 23392 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:13:38.175618 24948 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:38.206620 25013 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.207098 25013 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:38.205098 23389 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:38.207988 25013 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.207693 25027 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.214439 25027 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.191586 24950 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.215312 25027 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.217036 25027 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.195554 23388 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:38.210783 24948 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.230559 24948 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.209223 25013 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.239990 25014 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.240468 25014 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:38.241328 25014 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.243319 25014 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.267458 23466 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:38.267899 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 86600.7us
I20260709 12:13:38.269289 24956 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:38.271886 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.272810 25028 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:38.274667 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.275213 24975 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:38.275863 24975 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.276885 24975 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.259179 23216 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:38.304329 24995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53950 (request call id 509) took 1469 ms. Trace:
I20260709 12:13:38.304778 24995 rpcz_store.cc:276] 0709 12:13:36.834748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.836411 (+  1663us) service_pool.cc:224] Handling call
0709 12:13:37.102405 (+265994us) ts_tablet_manager.cc:934] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
0709 12:13:38.019503 (+917098us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:38.255179 (+235676us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:38.255383 (+   204us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:38.303252 (+ 47869us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:38.303655 (+   403us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:38.304056 (+   401us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:38.304119 (+    63us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":356,"tablet-copy.queue_time_us":237615,"thread_start_us":194122,"threads_started":2}
I20260709 12:13:38.309293 24995 tablet_copy_client.cc:806] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:38.310243 24995 tablet_copy_client.cc:670] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:38.314169 24912 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:38.315318 24995 tablet_copy_client.cc:538] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:13:38.334275 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59710.7us
I20260709 12:13:38.335284 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.336462 24950 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.339375 23313 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:38.339874 23313 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.341193 23313 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.342399 23313 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.326310 25028 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:38.357939 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.355608 23394 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:38.362985 23218 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.366494 23218 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.369267 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.375222 24968 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:38.375814 24968 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.360224 24950 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:13:38.376758 24968 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.377951 24950 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.378321 24950 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.364348 24972 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:38.388757 24972 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.389863 24972 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.402741 23465 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:38.402926 24961 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.403463 24961 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.402801 23470 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.406756 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:38.407411 23312 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:38.407783 23312 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:38.410626 23314 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:38.411240 23318 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:38.413393 24948 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.414228 24948 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.417260 24986 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:13:38.420571 23318 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:38.423071 23313 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:13:38.423637 23313 pending_rounds.cc:85] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 2
I20260709 12:13:38.423897 23313 pending_rounds.cc:107] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:13:38.424304 23313 raft_consensus.cc:2889] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:38.424810 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:38.425688 25016 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:38.382974 24950 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.428524 24950 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:38.432471 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.433844 24986 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:38.434962 24986 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.435222 25000 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:38.435632 25000 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.435454 24986 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.418674 24948 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.436642 25000 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.408876 23312 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.429289 24991 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:13:38.443521 25000 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:38.446256 24961 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.468254 23470 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:13:38.469604 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.480878 24975 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.481833 24999 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.481990 25027 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:38.482363 24999 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.483006 25027 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.483331 25027 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.484566 24999 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.485240 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:38.485906 23464 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:38.488655 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:38.489372 23464 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:38.490655 24999 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:38.492138 24953 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:38.492547 24953 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.456141 25017 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:13:38.493592 24953 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:38.484210 25027 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:38.540153 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 61668.7us
I20260709 12:13:38.568476 23216 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:13:38.575799 23216 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 5 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:38.577517 24942 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.578143 24986 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.578238 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.595085 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.597908 24953 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:38.599611 25035 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.162s	user 0.001s	sys 0.004s
I20260709 12:13:38.580288 24956 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.601521 24956 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.593662 24942 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.611974 24942 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:38.613338 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:38.613780 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.614048 23464 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:38.614394 24016 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:38.615497 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.617396 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:38.618223 23318 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:38.619134 24009 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:13:38.616613 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 75328.1us
I20260709 12:13:38.619227 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:38.627051 23313 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:38.628003 25027 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.629869 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:38.625610 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.636476 24975 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:38.647598 24975 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.647971 24975 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.624764 24942 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:38.652844 25028 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:38.653494 25028 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.654702 25028 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:38.656636 25028 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.662952 24995 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:38.663282 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.667407 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.668004 24942 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.668109 24010 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:38.668515 24942 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.670495 24011 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:38.674263 24956 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.638520 24991 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:38.615545 25017 consensus_queue.cc:1243] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:13:38.691418 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 59769.2us
I20260709 12:13:38.693351 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:38.694008 23316 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:13:38.694775 23312 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:38.695551 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:38.696166 23318 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:13:38.681282 24991 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.678046 24956 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.698727 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:38.702703 24013 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:38.703024 23316 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:13:38.702550 24991 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.710053 24967 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:38.710167 24956 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.722198 24967 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.740178 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.739787 23216 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.740406 23218 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 2 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:38.742591 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.743244 25011 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:13:38.744189 23267 auto_rebalancer.cc:629] Failed to schedule move for tablet 06d137914eb14302bb888a191819fb75: Illegal state: Replica 6253b834ffb744579b46fcede2fc0e91 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:13:38.744596 25011 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:38.745366 25011 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.745740 25011 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:38.746909 24956 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.750360 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:38.751245 23216 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:38.753710 24013 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:38.755012 23316 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:38.753311 25017 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:38.757184 25017 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.758309 25017 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.741056 25048 ts_tablet_manager.cc:933] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:38.737926 24967 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.776153 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:38.777034 23466 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:38.778954 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:38.779781 24980 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:38.780145 24980 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:38.780540 24980 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:38.784976 24013 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:38.787858 25050 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:38.768981 24975 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.797762 25014 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.809013 25014 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.810314 24975 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.799808 24942 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.811961 23388 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:38.812461 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:38.813091 23464 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:38.811975 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:38.814015 24013 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.814481 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.815348 25002 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:38.815716 25002 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.807521 24991 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.816035 25002 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:38.821662 24942 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:38.837023 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:38.818084 25014 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:38.832078 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:38.822108 25027 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:13:38.831030 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:38.878036 23312 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:13:38.879925 25051 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:13:38.882759 25011 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.883821 23314 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.886547 24013 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:38.891336 25011 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.895905 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:38.897011 25028 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:38.897504 25028 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:38.898325 25028 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.912649 23389 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:38.915050 23470 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:38.915786 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:38.931262 25048 tablet_copy_client.cc:323] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.195:33909
I20260709 12:13:38.955566 23388 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:38.958189 23465 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.960124 24967 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.958081 24991 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.965202 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:38.965830 24010 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.966589 24967 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.966351 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:38.967335 23312 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:38.968636 23464 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:38.970994 23314 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:38.972155 25002 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.972519 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:38.973842 24016 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:38.958200 23480 tablet_copy_service.cc:140] P da1e2fd157894634bf92d6c82c06ec64: Received BeginTabletCopySession request for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:60318)
I20260709 12:13:38.977802 23480 tablet_copy_service.cc:161] P da1e2fd157894634bf92d6c82c06ec64: Beginning new tablet copy session on tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:60318: session id = 54640ab8abeb4c949426d66c7e5aed87-ec4e3b5eed804f2a8c2a01e5d1ae5f53
I20260709 12:13:38.979785 23312 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:38.981055 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:38.981853 24890 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:38.982331 24890 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:38.983244 23465 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:38.985147 24890 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.944168 24980 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:38.989915 24980 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:38.990928 24010 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:38.997119 25002 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:38.997751 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:38.997584 25017 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.002341 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:39.006429 23466 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.006356 23980 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:39.011430 25014 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:39.012008 25014 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:39.002887 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:39.012996 25014 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.016078 23480 tablet_copy_source_session.cc:215] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:39.018652 23313 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:39.019819 25048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:39.019728 25017 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.024622 23466 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:39.025803 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:39.026916 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:39.027716 24016 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:39.034302 23389 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:39.035778 23394 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:39.036391 23316 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:39.037822 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:39.037753 23470 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:39.037895 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:39.041483 23468 raft_consensus.cc:2368] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:39.043253 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:39.044003 23321 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:39.047171 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:39.037139 25017 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:13:39.049707 25061 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:39.050637 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:39.051362 24012 raft_consensus.cc:2368] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:39.052155 23470 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:39.052330 23216 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.047827 23313 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:39.063545 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:39.061193 23313 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:39.067782 23465 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:39.068872 24961 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:39.069137 24995 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:39.070056 24942 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:39.075577 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:39.084044 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:39.084707 23287 leader_election.cc:400] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:39.085014 23287 leader_election.cc:403] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:39.073462 25002 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:39.092985 25015 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:39.101758 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.086056 24890 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:39.090991 24961 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.110361 24912 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:39.090194 24942 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:39.116447 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:39.081312 25017 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:39.117990 24890 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:39.126430 23287 consensus_peers.cc:597] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 87e270393fc841ddac0d9170b0bc6554. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:13:39.128942 23287 leader_election.cc:400] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:39.114818 24971 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.113247 24972 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:13:39.106388 25002 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.144768 23218 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:39.114914 24948 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:13:39.126111 24967 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:13:39.129695 25028 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:39.161125 24890 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.164139 24991 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.164644 24991 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.166728 24991 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.170588 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.171180 23470 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:39.171284 25053 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.138893 24942 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.174456 25002 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.132361 25061 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:13:39.176267 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.176676 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:39.181890 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:39.183521 23470 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:39.171748 25053 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.158922 24995 tablet_bootstrap.cc:492] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:39.150748 24971 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.216578 23318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:39.218071 23318 raft_consensus.cc:606] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:39.197713 24991 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:39.194592 25030 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.234429 23389 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:39.234454 24995 ts_tablet_manager.cc:1403] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.569s	user 0.027s	sys 0.033s
I20260709 12:13:39.229283 24991 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.233708 25030 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.291720 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:39.265038 24991 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:39.295750 25066 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:13:39.339092 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.339965 23465 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:39.345903 23468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:39.346567 23468 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:39.346923 23468 raft_consensus.cc:614] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:39.349747 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58155.5us
I20260709 12:13:39.334810 25002 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:39.355268 25002 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:39.378698 23313 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:39.379357 23313 raft_consensus.cc:606] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:39.394367 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:39.397976 25068 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:13:39.400799 25053 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.401626 24890 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:39.402168 24890 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.402789 25030 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.376765 24971 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.404320 24890 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.401494 25053 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.405946 24961 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:39.406458 25053 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.408025 23464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:39.408691 23464 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:39.409621 24971 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.410158 24971 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.412268 24971 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:39.413420 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71895.4us
I20260709 12:13:39.416364 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:39.416561 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.417186 23465 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:39.396476 25002 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.409068 25053 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.431156 24995 raft_consensus.cc:359] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:39.432050 24995 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:39.432727 24995 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:39.433821 25048 tablet_copy_client.cc:806] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:39.434484 25048 tablet_copy_client.cc:670] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
W20260709 12:13:39.412264 25063 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.205s	user 0.000s	sys 0.003s
I20260709 12:13:39.429899 24890 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:39.409093 23464 raft_consensus.cc:614] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:39.457079 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:39.460842 24980 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:39.461206 24980 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:39.461573 24980 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:39.485422 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.447921 24890 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.494601 24890 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:39.495767 24995 ts_tablet_manager.cc:1434] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.232s	user 0.001s	sys 0.008s
I20260709 12:13:39.452953 25027 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.497874 25027 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.500279 25027 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:39.501636 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45114.8us
I20260709 12:13:39.509008 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:39.511160 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:39.509953 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:39.454640 24986 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:39.531072 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:39.532606 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 200713us
W20260709 12:13:39.530855 25074 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:13:39.579723 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45467.2us
W20260709 12:13:39.580811 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49820.2us
I20260709 12:13:39.582397 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:39.591969 24013 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:39.528761 25002 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.611390 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:39.612300 23318 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:39.573622 24986 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.614282 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:39.601014 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:39.623545 24016 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:39.606519 25076 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:39.624804 24010 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:39.616881 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:39.635058 24991 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.638108 25069 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:39.638615 24986 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:39.646353 23318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:39.646950 23318 raft_consensus.cc:606] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:39.649237 24980 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.651365 24980 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.629556 25076 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.667764 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:39.668967 23395 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:39.660377 23314 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:13:39.674073 25080 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.144s	user 0.000s	sys 0.002s
I20260709 12:13:39.671231 25076 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.652504 25069 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.718880 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:39.752449 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:39.752908 23395 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:39.684821 24980 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:39.788504 25047 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:13:39.806311 24013 raft_consensus.cc:1217] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:13:39.807312 25008 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.808269 25008 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.810748 25008 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.812469 25070 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:39.813019 25070 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.818244 24972 raft_consensus.cc:993] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:39.818989 24972 raft_consensus.cc:1081] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:13:39.823132 24971 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:39.823917 24971 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.827281 25070 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:39.831202 23313 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:39.846689 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:39.855979 24008 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:39.858824 23312 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:39.862361 25014 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:39.863391 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:39.864507 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:39.866423 25017 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.867156 25017 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.881486 24971 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:39.885248 25017 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.892586 25050 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.790333 25077 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.790654 25084 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.622877 25016 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:39.785542 24980 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:39.923553 24980 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:39.894199 25050 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.892984 25070 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.894863 24968 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:39.905400 25077 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:39.902477 24986 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:39.934088 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 181746us
I20260709 12:13:39.916364 25030 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.955139 25050 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:39.988377 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 393241us
I20260709 12:13:39.990406 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.991127 23468 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:39.993088 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.996079 23464 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:39.993342 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:39.997222 23468 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:39.997761 23464 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:40.008476 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:40.009213 23315 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.011914 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.012928 23318 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:40.019032 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:40.020701 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:40.025679 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:40.029198 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:39.956540 24991 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.032166 24014 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:39.940025 24968 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.044204 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:40.046901 23468 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.004490 25086 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.002472 24980 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:40.055460 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:40.032740 24942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.006s
I20260709 12:13:39.943037 24986 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:40.063162 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 469431us
I20260709 12:13:40.022286 25011 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.031150 25090 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:39.988521 25030 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.030370 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:40.098093 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 96244.3us
I20260709 12:13:40.092504 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.111622 23315 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:13:40.061188 24942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.013s	sys 0.007s
I20260709 12:13:39.911484 25084 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:40.113629 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45822.1us
I20260709 12:13:40.113865 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:40.114425 23466 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:40.089150 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:40.116004 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:40.116647 24014 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.116132 23465 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.005590 25076 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.129114 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.067504 25070 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.091120 25002 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.021410 25088 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.109s
I20260709 12:13:40.195781 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.196760 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.197050 23468 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.197643 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.212865 23389 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:40.077154 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-22fc2096e78f47158bbb4bae3a474e2c received from {username='slave'} at 127.0.0.1:34554
W20260709 12:13:40.203909 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 74852.1us
I20260709 12:13:40.194087 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.217329 23387 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
W20260709 12:13:40.210949 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 85488.7us
I20260709 12:13:40.212332 23394 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:13:40.122112 25099 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 12:13:40.221230 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89345.1us
I20260709 12:13:40.223869 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.224498 23468 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:40.225734 23466 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:40.060994 25086 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.228094 25084 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.228119 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.228812 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.229147 23463 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:40.219424 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-22fc2096e78f47158bbb4bae3a474e2c on tablet 22fc2096e78f47158bbb4bae3a474e2c with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:40.231040 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:40.231721 24014 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.228194 25098 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:40.127643 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:40.131101 24899 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:40.236330 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.236992 23463 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:40.063737 24980 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:40.189639 25028 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.110050 25011 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.219467 23312 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:40.216192 25069 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.271672 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:40.279336 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43733.4us
I20260709 12:13:40.238692 25098 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.229055 25086 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.220916 24975 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:13:40.277911 25050 raft_consensus.cc:993] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:40.245723 24010 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.275123 25027 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:40.338219 25107 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:40.328429 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:40.342459 24014 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:40.298242 25098 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.302039 25077 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.246827 24899 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.355733 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:40.357394 24014 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:40.328521 24971 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:40.363174 25094 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.363780 25094 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.365144 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:40.349633 24942 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.351701 25030 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.278826 25028 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:40.349393 25027 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.360989 24971 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:40.404996 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.409823 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.412511 23396 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:40.417289 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.417974 23462 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:40.419691 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.372723 25090 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:40.402621 25048 tablet_copy_client.cc:538] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:40.396835 24968 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.413553 24971 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.441347 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.442600 23316 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:40.372584 24942 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.435240 25089 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:40.446255 25091 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:13:40.448724 25075 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:40.449198 25075 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.449587 25075 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.450459 24007 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:13:40.451458 24007 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:40.452354 24007 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:40.455829 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:40.456457 23462 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.431164 25011 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.432653 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.432068 24986 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:40.456483 25072 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:13:40.467159 24980 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.467718 25067 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:40.453722 25076 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.453729 25111 raft_consensus.cc:1081] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:40.444079 25101 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.458753 25089 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:40.482796 25105 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:13:40.474969 23313 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:40.482648 25090 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.501941 23462 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:40.502640 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:40.468973 25094 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.508831 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.488008 25067 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:40.519558 25104 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 12:13:40.511767 24890 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.017s	user 0.009s	sys 0.004s
I20260709 12:13:40.529745 25114 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:40.530396 25114 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.530923 25114 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.513414 25076 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:40.517037 25067 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.546866 25089 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.549750 25028 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.515419 25101 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.551059 24008 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:40.551992 23387 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:40.557152 25111 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:40.557765 25111 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.562659 25111 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.542318 25086 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:40.528630 24890 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:40.640411 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 368757us
I20260709 12:13:40.572984 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:40.655256 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 82377.9us
I20260709 12:13:40.558540 25076 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:40.660694 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.664915 23316 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:40.657940 25008 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:40.665737 25008 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.666204 25008 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:40.665781 24971 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:40.569947 25016 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.005s
I20260709 12:13:40.669829 23395 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:40.670805 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.671624 23319 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:40.659587 24010 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.657047 25101 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:40.587177 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.576303 25027 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.687115 25069 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:40.687701 25069 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.688412 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.693063 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.694594 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.677711 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.695181 23390 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:40.695703 23316 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:40.698022 23392 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:40.701814 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:40.703899 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:40.704461 23392 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.669597 25061 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:40.680771 25002 raft_consensus.cc:2764] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:40.659312 25086 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.707721 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:40.604264 25098 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.702945 23313 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.723434 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:40.749786 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:40.708031 25061 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.710690 25069 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.751082 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:40.766430 25081 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:40.767091 25081 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:40.767781 25081 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:40.771903 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.773468 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.774374 24971 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.755741 25061 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.775692 25098 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.776295 23393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:40.778373 23393 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:40.778999 23393 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.747308 24950 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:40.782025 25116 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:40.782588 25116 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:40.783023 25116 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:40.782048 24950 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.788684 23388 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:40.790400 24950 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.790596 24942 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.791919 25069 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:40.755519 25083 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:40.755522 25076 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.799808 25028 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:40.800401 25028 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:40.800932 24899 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.801501 24899 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:40.801940 24899 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.750801 25027 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:40.808210 25108 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:13:40.809010 24971 raft_consensus.cc:1064] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:40.808877 25075 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:40.810714 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61043.9us
I20260709 12:13:40.811455 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.812192 23462 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:40.813834 24950 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:40.810089 25027 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:40.814257 24950 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.814651 24950 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:40.783566 24890 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.819919 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:40.827562 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:40.828433 25096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.029s
W20260709 12:13:40.829252 25096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.000s	sys 0.029s
I20260709 12:13:40.830945 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:40.832095 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 402844us
I20260709 12:13:40.834450 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.836040 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.836423 23463 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:40.836921 23392 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:13:40.837285 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.838440 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:40.802456 25028 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:40.841841 23463 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:40.815835 25027 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:40.816524 25083 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:40.864970 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:40.868935 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:13:40.907125 23286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.007s	sys 0.033s
W20260709 12:13:40.911307 23286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.007s	sys 0.033s
I20260709 12:13:40.919478 25117 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:40.919682 25069 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:40.861797 25050 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.013s	sys 0.035s
I20260709 12:13:40.860644 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:40.920255 25117 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:40.986188 23466 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.028s	sys 0.017s
I20260709 12:13:41.386101 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.954301 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:41.178642 23469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.911s	user 0.011s	sys 0.025s
I20260709 12:13:41.412536 23468 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:41.412724 23469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.014s	sys 0.040s
I20260709 12:13:41.411614 23392 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
W20260709 12:13:41.383074 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 395810us
W20260709 12:13:41.223264 23389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.020s	sys 0.011s
W20260709 12:13:41.414558 23389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.029s	sys 0.018s
I20260709 12:13:40.920424 25069 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.418628 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:41.419562 23388 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:41.419029 23466 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.038s	sys 0.030s
I20260709 12:13:40.974506 23393 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.426278 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:40.860520 24008 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:13:41.385906 25075 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:41.383488 24007 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.930s	user 0.007s	sys 0.006s
W20260709 12:13:40.955454 25080 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:13:41.453063 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 262) took 1425 ms. Trace:
W20260709 12:13:41.307379 25016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.013s	sys 0.013s
W20260709 12:13:41.453567 25116 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.670s	user 0.009s	sys 0.003s
W20260709 12:13:40.944607 25050 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.013s	sys 0.035s
W20260709 12:13:41.453883 25016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.013s	sys 0.014s
I20260709 12:13:41.453976 25116 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.454511 25095 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:41.454594 25016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:43740 (request call id 13) took 2707 ms. Trace:
I20260709 12:13:41.309298 25073 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:41.435213 25053 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:41.444895 24948 raft_consensus.cc:993] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:41.446777 25048 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:41.448393 23468 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:41.142731 25070 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.452219 25075 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:41.452952 24007 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.428053 25121 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.245003 24899 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:41.225864 25126 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.142742 25108 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.456135 23393 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:41.459820 23287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.018s	sys 0.012s
W20260709 12:13:41.489146 23287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.018s	sys 0.012s
W20260709 12:13:41.489567 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 584620us
I20260709 12:13:40.953375 23464 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:13:41.462060 23391 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.023s	sys 0.029s
W20260709 12:13:41.493636 23391 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.023s	sys 0.029s
I20260709 12:13:41.500454 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:41.418565 25086 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:41.456346 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:41.502637 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:41.456048 23395 rpcz_store.cc:276] 0709 12:13:40.027938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.028976 (+  1038us) service_pool.cc:224] Handling call
0709 12:13:41.452809 (+1423833us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54003453d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:41.452963 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470557568}
I20260709 12:13:41.425305 25099 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:13:41.397565 25092 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.006s	sys 0.037s
W20260709 12:13:41.506707 25092 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.106s	user 0.006s	sys 0.038s
I20260709 12:13:41.506237 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:41.461263 23388 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:41.224647 25083 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.523288 23394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.017s	sys 0.051s
W20260709 12:13:41.551055 23394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.019s	sys 0.056s
I20260709 12:13:41.472525 25070 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.463811 25073 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:41.520885 25126 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.477394 25090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.011s	sys 0.004s
I20260709 12:13:41.562251 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:41.482542 25125 raft_consensus.cc:1081] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:41.478183 25075 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.481813 25121 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.463057 25053 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.559504 23392 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:41.486196 24899 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.455109 25095 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.599257 24899 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.601126 25095 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:41.581681 25076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.022s	sys 0.013s
W20260709 12:13:41.601972 25076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.023s	sys 0.014s
W20260709 12:13:41.578581 25090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s	user 0.011s	sys 0.004s
I20260709 12:13:41.576274 25073 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:41.543874 25108 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:41.614485 23359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.023s
I20260709 12:13:41.565984 25111 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:41.621923 23359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.000s	sys 0.024s
I20260709 12:13:41.622252 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:41.581792 25123 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:41.622885 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:41.685421 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:41.611280 25089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.902s	user 0.010s	sys 0.031s
I20260709 12:13:41.606359 25099 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:41.627581 23286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.021s	sys 0.024s
W20260709 12:13:41.695232 23286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.021s	sys 0.024s
W20260709 12:13:41.685729 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 188648us
I20260709 12:13:41.611462 25116 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:41.685762 25067 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.121s	user 0.010s	sys 0.002s
I20260709 12:13:41.691332 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:41.693908 25089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.010s	sys 0.031s
I20260709 12:13:41.751441 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:41.731371 24014 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:41.678953 25101 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:41.688160 25090 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:41.760396 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 198226us
I20260709 12:13:41.701298 25079 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.773077 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:41.723915 24899 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:41.786051 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:41.786717 23319 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:41.784068 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:41.784299 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2058) took 1059 ms. Trace:
I20260709 12:13:41.725165 25095 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.755642 25096 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:41.455101 25016 rpcz_store.cc:276] 0709 12:13:38.747582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.747878 (+   296us) service_pool.cc:224] Handling call
0709 12:13:38.764810 (+ 16932us) log.cc:844] Serialized 513 byte log entry
0709 12:13:38.975035 (+210225us) spinlock_profiling.cc:243] Waited 966 us on lock 0x7b18000ad200. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:13:38.989305 (+ 14270us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18000ad200. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:13:41.454491 (+2465186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7099,"raft.queue_time_us":7915,"raft.run_cpu_time_us":2238,"raft.run_wall_time_us":36379,"spinlock_wait_cycles":9958528,"thread_start_us":51322,"threads_started":1,"wal-append.queue_time_us":108162,"wal-append.run_cpu_time_us":1787,"wal-append.run_wall_time_us":1014207}
I20260709 12:13:41.686717 25111 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.795353 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:41.803858 23316 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:41.776784 25069 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:41.796890 25101 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:41.755510 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:41.685173 25123 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.782137 25061 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.006s	user 0.005s	sys 0.007s
I20260709 12:13:41.754954 25050 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.787989 23391 rpcz_store.cc:276] 0709 12:13:40.718055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.718604 (+   549us) service_pool.cc:224] Handling call
0709 12:13:40.718940 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.719213 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:41.494116 (+774903us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.777157 (+283041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.777213 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:41.777250 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.777311 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.777424 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4494,"prepare.queue_time_us":1006051,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":4541,"thread_start_us":1021180,"threads_started":2}
I20260709 12:13:41.706355 25099 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.815116 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2117) took 1572 ms. Trace:
I20260709 12:13:41.850610 25155 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:41.850728 23469 rpcz_store.cc:276] 0709 12:13:40.230728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.233118 (+  2390us) service_pool.cc:224] Handling call
0709 12:13:40.233381 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.233595 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:13:41.419952 (+1186357us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.802833 (+382881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.802887 (+    54us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:41.802928 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.802995 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.803106 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1316422,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":54,"thread_start_us":1548896,"threads_started":3,"wal-append.queue_time_us":331059}
I20260709 12:13:41.779924 25089 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.852347 25155 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:41.852501 25099 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:41.852577 25089 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.779887 24991 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.819528 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:41.821794 25108 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:41.828547 25073 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.860656 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:41.834966 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:41.857577 25121 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:41.836125 23388 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
I20260709 12:13:41.860810 24991 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.777453 25090 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.846271 25131 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:13:41.813287 25075 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:41.759492 25067 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.854185 25096 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:41.857386 25089 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:41.857034 25101 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.850231 25156 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.831657 25069 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.786670 25070 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:41.800433 25095 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.849445 25008 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.183s	user 0.006s	sys 0.007s
I20260709 12:13:41.830111 23393 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:41.786072 25079 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.804719 25113 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:41.866740 25099 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:41.867655 25149 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:13:41.877528 25135 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 12:13:41.865779 23388 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:41.869537 23313 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:41.858199 25050 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:41.884792 24950 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.070s	user 0.012s	sys 0.001s
I20260709 12:13:41.872713 25096 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:41.849184 25061 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.875963 25108 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:41.903167 23388 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:41.904273 24011 raft_consensus.cc:2368] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
W20260709 12:13:41.904492 25133 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:13:41.905419 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:41.932566 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 260) took 1476 ms. Trace:
I20260709 12:13:41.937018 23390 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:41.884408 25126 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:41.932760 24950 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.941968 25160 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:41.942296 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 270) took 1242 ms (client timeout 1655 ms). Trace:
I20260709 12:13:41.936889 23466 rpcz_store.cc:276] 0709 12:13:40.455802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.455973 (+   171us) service_pool.cc:224] Handling call
0709 12:13:40.456288 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.456623 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.456653 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.459137 (+  2484us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.421034 (+961897us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:41.421088 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.421177 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.932342 (+511165us) raft_consensus.cc:1685] finished
0709 12:13:41.932366 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.932504 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17717,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7612,"thread_start_us":960118,"threads_started":1,"wal-append.queue_time_us":1120449}
I20260709 12:13:41.950726 23318 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:41.892970 25123 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:41.950798 23390 rpcz_store.cc:269] 0709 12:13:40.695689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.905053 (+1209364us) service_pool.cc:224] Handling call
0709 12:13:41.937901 (+ 32848us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:41.949225 23388 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.889075 25069 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:41.943259 25108 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:41.931449 25070 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:41.957651 25150 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:41.962707 23388 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:41.965982 25090 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:41.966745 25144 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:13:41.905968 25099 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:41.970458 25130 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.072s	user 0.000s	sys 0.005s
I20260709 12:13:41.971891 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1473) took 1545 ms. Trace:
I20260709 12:13:41.972302 23315 rpcz_store.cc:276] 0709 12:13:40.426126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.426303 (+   177us) service_pool.cc:224] Handling call
0709 12:13:40.437089 (+ 10786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.437478 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.437521 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.457183 (+ 19662us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.823307 (+366124us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:40.823366 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.823463 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.971621 (+1148158us) raft_consensus.cc:1685] finished
0709 12:13:41.971649 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.971805 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":492,"raft.queue_time_us":16843,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6081,"thread_start_us":349521,"threads_started":1,"wal-append.queue_time_us":1309053}
I20260709 12:13:41.931138 25113 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.961578 25126 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:41.964301 25092 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:41.999150 25154 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:13:42.008754 25113 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.011977 25079 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:41.888015 25156 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.038121 24995 ts_tablet_manager.cc:933] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:42.038741 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 415) took 1755 ms. Trace:
I20260709 12:13:41.968439 25070 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.040777 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:42.096256 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55606.7us
I20260709 12:13:42.084276 25002 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.998145 25111 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.102784 25141 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:42.099182 25132 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:13:42.091061 24972 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:42.007530 25126 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.014690 25096 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.139807 25096 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:42.144165 25099 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:42.144642 25138 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:13:41.983474 25150 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.105841 24991 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.113296 25067 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:42.159170 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 327974us
I20260709 12:13:41.897784 25008 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.114013 25111 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:42.155015 23319 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:42.108042 25002 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:41.962918 25160 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.154080 25095 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:42.159780 25114 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.628s	user 0.005s	sys 0.009s
I20260709 12:13:42.147924 24972 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:42.177614 25142 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:13:42.172392 25050 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.192853 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2125) took 1341 ms. Trace:
I20260709 12:13:42.193244 23462 rpcz_store.cc:276] 0709 12:13:40.851541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.852085 (+   544us) service_pool.cc:224] Handling call
0709 12:13:42.192173 (+1340088us) spinlock_profiling.cc:243] Waited 971 us on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:42.192366 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.192533 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.192559 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.192600 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:42.192630 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.192685 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.192797 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2233088}
I20260709 12:13:42.202153 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.202915 23316 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:42.084483 25092 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.206200 25008 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.206851 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 261) took 1703 ms. Trace:
I20260709 12:13:42.179401 25061 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.174734 24975 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:42.185175 25150 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.206302 25092 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:42.210940 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1241) took 1799 ms (client timeout 1758 ms). Trace:
I20260709 12:13:42.090834 24009 rpcz_store.cc:276] 0709 12:13:40.282909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.702891 (+419982us) service_pool.cc:224] Handling call
0709 12:13:42.021089 (+1318198us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:42.037980 (+ 16891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.038236 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.038279 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.038359 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:42.038402 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.038463 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.038639 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2851712}
I20260709 12:13:42.201382 25125 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:42.214213 23470 rpcz_store.cc:276] 0709 12:13:40.503767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.504430 (+   663us) service_pool.cc:224] Handling call
0709 12:13:40.504825 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.505112 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.505150 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.506058 (+   908us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.887556 (+381498us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:40.887612 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.887706 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.206654 (+1318948us) raft_consensus.cc:1685] finished
0709 12:13:42.206675 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.206797 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":359031,"threads_started":1,"wal-append.queue_time_us":1007167}
I20260709 12:13:42.201195 25050 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:42.176728 25111 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.211229 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:42.236716 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63403.6us
I20260709 12:13:42.237352 25008 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:42.191041 24950 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.263980 25148 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:42.221896 23387 rpcz_store.cc:269] 0709 12:13:40.404001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.404764 (+   763us) service_pool.cc:224] Handling call
0709 12:13:42.203634 (+1798870us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:42.203742 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336317696}
I20260709 12:13:42.208370 25108 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:42.193202 25114 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.271775 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:42.272534 24014 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:42.276149 25147 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:13:42.276820 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.278728 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:42.280143 25002 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:42.268848 25095 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:42.281670 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.281447 25095 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.268503 25159 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:42.282486 25159 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.282830 25159 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.254531 25126 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.262122 23316 raft_consensus.cc:2368] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
W20260709 12:13:42.289625 25153 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:13:42.290289 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.291651 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.292373 23316 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:42.214926 25069 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.154232 25141 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.261469 25075 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:42.222287 24975 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.286758 25088 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:42.332799 23396 raft_consensus.cc:2368] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:42.268615 25076 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:42.297472 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.334756 25156 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.337702 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.300077 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.264539 25148 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.291440 25090 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:42.287590 25095 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.344617 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.261785 25050 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.363534 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.340260 25070 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.351176 25160 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.350569 24971 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.360209 24016 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:42.288946 25030 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:42.347170 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.259979 25125 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.374796 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:42.353096 24975 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.334139 25088 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.380537 23317 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:42.371408 24014 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:42.331516 25075 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.380929 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:42.385764 24016 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.340894 25069 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:42.375430 25081 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.607s	user 0.014s	sys 0.002s
W20260709 12:13:42.388876 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.391251 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.388993 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.343235 25076 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.409798 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:42.402858 25016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.014s	sys 0.030s
I20260709 12:13:42.423977 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:42.440711 23387 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.429098 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.441815 23393 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:42.429543 23388 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:42.442922 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.436655 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:42.358126 25090 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:42.438896 23320 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.016s	sys 0.055s
I20260709 12:13:42.413671 24016 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:42.433537 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:13:42.443707 23320 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.016s	sys 0.055s
I20260709 12:13:42.443871 23321 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:42.444639 23391 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.450958 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:42.451750 24014 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.448863 23321 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:13:42.440270 25016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.014s	sys 0.030s
I20260709 12:13:42.462038 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.462955 23360 leader_election.cc:400] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:42.463479 23360 leader_election.cc:403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:13:42.437983 25008 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:42.462901 23393 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:42.382594 25030 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.464757 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.392223 25069 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.440542 24972 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.442337 25161 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:42.473723 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.011s	sys 0.043s
W20260709 12:13:42.477770 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.012s	sys 0.043s
W20260709 12:13:42.478423 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 778792us
I20260709 12:13:42.456881 23321 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:42.476016 23287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.749s (SENT)
I20260709 12:13:42.480711 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:42.509140 23321 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:42.517510 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:42.474144 25160 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:42.517270 25077 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.073s	user 0.017s	sys 0.043s
I20260709 12:13:42.440665 25081 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:42.636264 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118843us
I20260709 12:13:42.634035 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:42.637311 25077 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.018s	sys 0.043s
I20260709 12:13:42.472891 25008 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.642015 25177 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:42.645483 25174 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:13:42.639904 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 161691us
I20260709 12:13:42.647199 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:42.502480 25088 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.519075 25114 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.655179 25166 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:42.637107 25168 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:42.655689 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.011s	sys 0.032s
W20260709 12:13:42.657496 24995 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.044s	sys 0.087s
I20260709 12:13:42.667248 25165 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:42.635468 25111 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.518818 25117 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.653208 25095 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:42.653048 25126 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.637645 25160 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.500597 25161 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.753878 24014 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
W20260709 12:13:42.712757 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.011s	sys 0.033s
I20260709 12:13:42.715991 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.739265 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 515) took 1898 ms. Trace:
W20260709 12:13:42.787637 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 140503us
W20260709 12:13:42.794615 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.784472 24015 rpcz_store.cc:276] 0709 12:13:40.840341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.840468 (+   127us) service_pool.cc:224] Handling call
0709 12:13:42.290028 (+1449560us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:42.290222 (+   194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.290580 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.290611 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.291064 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.421629 (+130565us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:42.421687 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.421780 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.735221 (+313441us) raft_consensus.cc:1685] finished
0709 12:13:42.735251 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.739173 (+  3922us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37540,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5431,"spinlock_wait_cycles":1201159424,"thread_start_us":130171,"threads_started":1,"wal-append.queue_time_us":355294}
I20260709 12:13:42.788703 23393 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:42.788556 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:42.818096 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.805004 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:42.811419 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.820168 23470 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.656189 24972 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:42.698843 24995 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.044s	sys 0.087s
I20260709 12:13:42.668622 25177 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:42.835359 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.734906 25166 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:42.761384 25090 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.737833 25008 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:42.761868 25125 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.838763 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:42.843752 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:42.848846 23389 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:42.840272 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:42.844092 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.838093 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1256) took 1452 ms. Trace:
I20260709 12:13:42.839689 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.850049 23390 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.713166 25168 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:42.861061 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.717998 25165 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.836449 25169 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:42.873211 25030 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:42.850075 23392 rpcz_store.cc:276] 0709 12:13:41.385615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.502350 (+116735us) service_pool.cc:224] Handling call
0709 12:13:42.837933 (+1335583us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b54000a8554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:42.838043 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128555264}
I20260709 12:13:42.837060 25177 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:42.840639 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.817849 24975 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:42.848435 23466 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:42.870788 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 213238us
I20260709 12:13:42.849164 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.873310 25165 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.856277 24995 tablet_copy_client.cc:323] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:13:42.837164 25156 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:42.877761 23470 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:42.840471 25166 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.838102 25081 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.880705 23393 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:42.895802 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.901746 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:42.902870 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2054) took 2212 ms. Trace:
I20260709 12:13:42.903057 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.876482 25070 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.831280 25095 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.878820 24975 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.904284 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.873677 25169 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.877100 25168 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.831053 25126 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.931577 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.836712 24972 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:42.916170 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:42.892055 23461 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:42.917455 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.917721 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.938493 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.912034 25090 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:42.890775 25148 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.923991 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.927770 23389 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
W20260709 12:13:42.928424 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 452440us
I20260709 12:13:42.931270 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:42.931309 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:42.910822 23466 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:42.945546 23391 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:42.946050 23315 raft_consensus.cc:2368] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:42.900724 24991 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:42.934837 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:42.914968 25070 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:42.947944 25172 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:13:42.949056 25175 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:42.949712 25175 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:42.949853 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.950156 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.950948 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.901875 25069 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:42.951844 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1190) took 1163 ms. Trace:
I20260709 12:13:42.944986 23469 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:42.918668 23462 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:13:42.950661 25175 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:42.915195 24975 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:42.952997 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:42.935765 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 147103us
I20260709 12:13:42.939706 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.963320 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.959163 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.963901 24015 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:42.964020 23391 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:42.960040 23319 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:42.954236 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.960557 23389 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:42.962818 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.965467 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:42.966629 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.968369 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.968986 23315 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:42.969297 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.912263 25088 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:13:42.979081 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:42.981699 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:42.986768 23470 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:42.989243 23387 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.548s	user 0.007s	sys 0.004s
I20260709 12:13:42.989751 23387 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:42.999719 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.949079 25070 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.005898 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:43.006807 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.003881 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:43.020484 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.021376 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:43.023649 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.002763 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:43.002570 23469 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:42.958920 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.952252 23314 rpcz_store.cc:276] 0709 12:13:41.788564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.794148 (+  5584us) service_pool.cc:224] Handling call
0709 12:13:42.950786 (+1156638us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:42.951757 (+   971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43446144}
I20260709 12:13:43.026103 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:42.987022 25015 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:43.028844 23470 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:43.030230 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:43.002553 25159 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.709s	user 0.008s	sys 0.011s
I20260709 12:13:43.033466 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:42.987989 25188 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.044560 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:43.039352 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:43.045738 23392 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:43.046855 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:43.040738 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:43.047940 23315 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:43.042191 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:43.051170 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:42.962141 24991 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.005635 25166 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:43.037286 23469 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:43.017724 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:42.995761 25163 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:43.068192 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:43.053592 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:43.027776 25111 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:43.040071 25167 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:43.088263 25169 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:43.088634 25167 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:43.032634 25156 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:43.037071 25159 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.092185 25179 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.092739 25179 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.037812 25090 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:43.093358 25069 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:43.054217 25088 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:43.098894 25183 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:13:43.100090 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1191) took 1311 ms. Trace:
W20260709 12:13:43.100701 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 102737us
I20260709 12:13:43.060930 25016 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:43.103956 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50441.4us
I20260709 12:13:43.058436 25166 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:43.090632 23393 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.090368 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.089071 25167 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.051941 25015 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:43.095127 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44053.9us
I20260709 12:13:43.095314 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.056278 25155 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:43.042606 23464 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:43.102663 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:43.090344 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:43.143501 25166 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.158236 25117 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:43.089354 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.080857 25111 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:43.097844 25161 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.097769 25088 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.158818 25117 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:43.137049 25164 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:43.055900 25188 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.108150 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.090238 24009 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:43.149900 25015 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.145466 25190 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.198802 24008 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:43.159816 25016 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:43.097388 25090 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.216899 23316 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:43.255640 23316 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:43.192015 23461 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:43.162947 25155 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:43.190910 25111 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.199842 25088 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.250236 24007 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:43.237775 23314 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:43.218789 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.283100 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
W20260709 12:13:43.283785 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1447) took 2403 ms (client timeout 1988 ms). Trace:
W20260709 12:13:43.284226 23463 rpcz_store.cc:269] 0709 12:13:40.880781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.881995 (+  1214us) service_pool.cc:224] Handling call
0709 12:13:43.218570 (+2336575us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:43.283666 (+ 65096us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5090688}
I20260709 12:13:43.257599 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:43.285569 25155 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.289093 25073 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.216796 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:43.259260 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.219291 25126 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:43.264204 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.232537 25164 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:43.261483 23316 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.298516 23316 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.276376 25159 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.206949 25117 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.298724 25164 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.303097 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53940 (request call id 413) took 3060 ms. Trace:
I20260709 12:13:43.246999 25190 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.231650 25156 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.304560 23464 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:43.312680 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.328572 25075 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.346180 25076 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:43.354048 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1185) took 1570 ms. Trace:
I20260709 12:13:43.281422 25016 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.303752 24007 rpcz_store.cc:276] 0709 12:13:40.232804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.270444 (+ 37640us) service_pool.cc:224] Handling call
0709 12:13:43.246428 (+2975984us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b4c002848c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7940a20 00007ff0a7951a29 00007ff0a795213f 00007ff0a79162ab 00007ff0a7943f3c 00007ff0a79443fe 00007ff0a7bd188b 00007ff0a95d0495 00007ff0a962130f 00007ff0a966ba55 00007ff0ae63bcd0
0709 12:13:43.282393 (+ 35965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18972032}
I20260709 12:13:43.354624 23318 rpcz_store.cc:276] 0709 12:13:41.783694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.783809 (+   115us) service_pool.cc:224] Handling call
0709 12:13:43.353765 (+1569956us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:43.353958 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374247296}
I20260709 12:13:43.363358 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:43.370890 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 228723us
I20260709 12:13:43.251830 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 274) took 1559 ms. Trace:
I20260709 12:13:43.373150 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:43.371912 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.373423 23395 rpcz_store.cc:276] 0709 12:13:41.692637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.692754 (+   117us) service_pool.cc:224] Handling call
0709 12:13:41.693075 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.884512 (+191437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.884555 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.884620 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:41.884664 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.251484 (+1366820us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400345dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:43.251528 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.251659 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439732480}
I20260709 12:13:43.100420 23320 rpcz_store.cc:276] 0709 12:13:41.788850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.788969 (+   119us) service_pool.cc:224] Handling call
0709 12:13:41.789291 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.448784 (+659493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.448840 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.458758 (+  9918us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.821127 (+362369us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:42.821161 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.948487 (+127326us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b540001de54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:42.948542 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.099815 (+151273us) raft_consensus.cc:1685] finished
0709 12:13:43.099835 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.100035 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":825,"raft.queue_time_us":17605,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":107540352,"thread_start_us":952358,"threads_started":2,"wal-append.queue_time_us":430437}
I20260709 12:13:43.375895 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:43.379309 23390 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.015s
I20260709 12:13:43.379841 23390 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:43.384174 23315 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:13:43.386435 23321 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.869s	user 0.022s	sys 0.000s
I20260709 12:13:43.386929 23321 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:43.387473 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1490) took 1234 ms. Trace:
I20260709 12:13:43.381305 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:43.395535 25171 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:43.396332 25171 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:43.397634 25171 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.404624 24009 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:43.325352 25126 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.409144 23466 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:43.387868 23321 rpcz_store.cc:276] 0709 12:13:42.153969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.436322 (+282353us) service_pool.cc:224] Handling call
0709 12:13:43.387407 (+951085us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:43.503527 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 400923us
I20260709 12:13:43.504443 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:43.511519 25188 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:43.514695 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 420351us
I20260709 12:13:43.518266 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 87e270393fc841ddac0d9170b0bc6554 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:13:43.518918 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 87e270393fc841ddac0d9170b0bc6554 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-87e270393fc841ddac0d9170b0bc6554
I20260709 12:13:43.519011 25095 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.530084 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:43.385946 25114 raft_consensus.cc:2764] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:13:43.541720 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:43.409451 25163 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:43.501562 25075 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.505709 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:43.373183 25069 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.556254 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.545511 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:43.565092 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:43.537967 25053 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.546553 25188 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.548743 25163 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:43.548097 25169 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:13:43.593475 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89130.3us
I20260709 12:13:43.550554 25186 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.514032 25126 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.503335 25199 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.600486 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:43.566359 24972 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.551093 25157 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:43.631558 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67089.5us
I20260709 12:13:43.537587 25198 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.600206 25163 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:43.559301 25075 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.570245 25191 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.601017 24008 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:43.556514 25200 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:13:43.628346 24007 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:13:43.628698 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 516) took 2761 ms. Trace:
I20260709 12:13:43.632236 25079 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:43.600239 25188 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.606309 25169 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.648006 25157 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.633702 25199 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.698093 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:43.698945 25079 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:43.699548 25079 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:43.708390 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:43.708529 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.708529 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.711501 25198 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:43.712095 25198 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.712584 25198 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:43.712193 23467 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:13:43.733798 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.608418 25186 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:43.740242 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 139835us
W20260709 12:13:43.742933 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 178808us
I20260709 12:13:43.743089 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:43.744343 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:43.745234 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:43.693578 25076 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:43.695312 25157 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.747599 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:13:43.748468 23393 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.610s	user 0.027s	sys 0.030s
I20260709 12:13:43.750849 23393 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:43.752518 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:43.755549 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:43.755908 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:43.756091 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:43.757664 23395 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:43.750756 25196 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:43.750216 25030 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:43.675177 24010 rpcz_store.cc:276] 0709 12:13:40.844811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.849087 (+  4276us) service_pool.cc:224] Handling call
0709 12:13:41.418318 (+569231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.617893 (+199575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.621413 (+  3520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.678690 (+ 57277us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:41.685387 (+  6697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.578974 (+1893587us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400083754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:43.591011 (+ 12037us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.605851 (+ 14840us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430891264}
I20260709 12:13:43.828955 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:43.830631 25096 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.690s	user 0.041s	sys 0.012s
I20260709 12:13:43.757539 25179 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.759634 25076 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:43.757228 25202 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:43.833987 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.795797 25190 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:43.759459 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.739810 25095 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:13:43.845469 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 98654.8us
I20260709 12:13:43.829336 25030 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.759462 25070 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 12:13:43.854310 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.851724 25197 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:43.861526 23466 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:43.865893 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 113482us
I20260709 12:13:43.859318 25030 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:43.872155 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:43.845891 25202 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.863544 25069 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:43.860983 23464 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:43.881500 23436 connection.cc:256] RPC call timeout handler was delayed by -1.0974200590001146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 12:13:43.866164 25197 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.829788 25201 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:13:43.925650 25191 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:43.879112 25048 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:13:43.934227 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:43.876580 25190 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.871554 25126 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.935002 25076 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.936991 25188 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.928992 25202 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:43.935144 25197 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:43.937528 25214 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:43.846200 25079 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.963315 25214 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:43.944898 25186 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:43.963212 25209 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:43.944977 25190 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:43.950201 25188 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:43.972462 25075 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:13:44.055823 25053 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.994618 25209 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:43.944749 25191 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.010630 25169 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:43.996469 25188 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:44.075131 25209 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.076423 25053 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.098951 25053 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:44.101322 25015 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.865s	user 0.022s	sys 0.013s
W20260709 12:13:44.104177 24975 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.127s	user 0.018s	sys 0.011s
I20260709 12:13:44.102829 25015 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.104658 24975 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:44.102029 25166 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.935s	user 0.017s	sys 0.014s
I20260709 12:13:44.076994 25199 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.115664 25214 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.102375 25190 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.119400 25166 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:44.204188 25048 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 12:13:44.208207 25176 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.547s	user 0.004s	sys 0.001s
W20260709 12:13:44.219177 25165 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.322s	user 0.024s	sys 0.050s
I20260709 12:13:44.211668 24975 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:44.222337 25088 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.940s	user 0.009s	sys 0.011s
I20260709 12:13:44.222817 25088 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.224573 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.225054 25088 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.219637 25165 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.228010 25165 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.229139 25191 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:44.216843 25168 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.279s	user 0.005s	sys 0.016s
W20260709 12:13:44.233846 25163 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.568s	user 0.016s	sys 0.026s
I20260709 12:13:44.233803 25168 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.234364 25163 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:44.242362 25198 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.529s	user 0.015s	sys 0.012s
I20260709 12:13:44.242826 25198 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.247129 25163 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:44.249478 25167 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.105s	user 0.008s	sys 0.005s
I20260709 12:13:44.249953 25167 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:44.251029 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 320424us
I20260709 12:13:44.241008 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.253744 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:44.259872 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.260667 23464 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
W20260709 12:13:44.264833 25157 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.503s	user 0.021s	sys 0.038s
W20260709 12:13:44.237027 25117 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.933s	user 0.018s	sys 0.004s
I20260709 12:13:44.252962 25198 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:44.245546 25177 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.351s	user 0.020s	sys 0.015s
I20260709 12:13:44.279373 25167 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.273554 25015 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.265415 25157 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:44.280871 25219 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:13:44.270961 25189 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:44.281627 25209 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.276947 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.276959 25117 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.282284 25157 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.282866 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:44.273497 25166 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.287719 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:44.266592 25168 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.294234 25048 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:44.295303 25048 tablet_bootstrap.cc:492] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:44.296075 25048 ts_tablet_manager.cc:1403] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 2.850s	user 0.035s	sys 0.041s
I20260709 12:13:44.283641 25188 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.297610 23464 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:44.308588 23331 tablet_copy_source_session.cc:215] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:13:44.302136 25117 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.272550 25202 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.322297 25160 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.284001 25177 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.331260 25141 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:44.327541 25111 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.048s	user 0.028s	sys 0.025s
I20260709 12:13:44.313815 25030 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.336155 25202 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.333381 25111 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.331923 25209 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.333627 25197 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.353557 25177 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.354444 25111 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.355578 25030 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.356539 25197 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.357335 25048 raft_consensus.cc:359] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:44.358572 25048 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:13:44.359318 25048 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:44.360323 23314 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:44.361268 25048 ts_tablet_manager.cc:1434] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.002s
W20260709 12:13:44.372575 23393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 275) took 1442 ms (client timeout 1662 ms). Trace:
W20260709 12:13:44.373131 23393 rpcz_store.cc:269] 0709 12:13:42.930903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.931025 (+   122us) service_pool.cc:224] Handling call
0709 12:13:44.372268 (+1441243us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001964d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.372487 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332844800}
I20260709 12:13:43.551455 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1472) took 3119 ms. Trace:
I20260709 12:13:44.374975 23312 rpcz_store.cc:276] 0709 12:13:40.417355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.418444 (+  1089us) service_pool.cc:224] Handling call
0709 12:13:42.167017 (+1748573us) spinlock_profiling.cc:243] Waited 4.51 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:42.183496 (+ 16479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.217874 (+ 34378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.220121 (+  2247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.250381 (+ 30260us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:42.251785 (+  1404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.503363 (+1251578us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54002747d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:43.514149 (+ 10786us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.536967 (+ 22818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76767232}
I20260709 12:13:44.375921 23464 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:44.376781 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.377645 23312 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:44.380003 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.389209 23312 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:44.390395 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.391255 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1504) took 1330 ms. Trace:
I20260709 12:13:44.391666 23312 rpcz_store.cc:276] 0709 12:13:43.061189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.390087 (+1328898us) service_pool.cc:224] Handling call
0709 12:13:44.391197 (+  1110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.392746 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.393574 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1503) took 1332 ms. Trace:
I20260709 12:13:44.394151 23312 rpcz_store.cc:276] 0709 12:13:43.060874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.392478 (+1331604us) service_pool.cc:224] Handling call
0709 12:13:44.393520 (+  1042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.395165 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.396588 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.397576 23312 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:13:44.398030 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 351) took 2016 ms. Trace:
I20260709 12:13:44.398428 23312 rpcz_store.cc:276] 0709 12:13:42.382147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.396298 (+2014151us) service_pool.cc:224] Handling call
0709 12:13:44.397977 (+  1679us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.400249 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 338) took 2360 ms. Trace:
I20260709 12:13:44.400655 23387 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:44.402310 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 339) took 2307 ms. Trace:
I20260709 12:13:44.402549 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 281) took 1460 ms. Trace:
I20260709 12:13:44.402719 23315 rpcz_store.cc:276] 0709 12:13:42.094879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.401547 (+2306668us) service_pool.cc:224] Handling call
0709 12:13:44.401837 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.402004 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.402030 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.402073 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.402105 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.402155 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.402252 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.402974 23387 rpcz_store.cc:276] 0709 12:13:42.942159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.991666 (+ 49507us) service_pool.cc:224] Handling call
0709 12:13:42.991957 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.401263 (+1409306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.402309 (+  1046us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002af8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:44.402492 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3239262976}
I20260709 12:13:44.404755 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 340) took 2309 ms. Trace:
I20260709 12:13:44.405423 23315 rpcz_store.cc:276] 0709 12:13:42.095144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.404005 (+2308861us) service_pool.cc:224] Handling call
0709 12:13:44.404284 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.404444 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.404471 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.404522 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:44.404553 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.404604 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.404705 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.400645 23312 rpcz_store.cc:276] 0709 12:13:42.040128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.399333 (+2359205us) service_pool.cc:224] Handling call
0709 12:13:44.399677 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.399876 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.399910 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.399964 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.400000 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.400066 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.400192 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.408291 23312 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:44.408957 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 342) took 2305 ms. Trace:
I20260709 12:13:44.409298 23312 rpcz_store.cc:276] 0709 12:13:42.103773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.407800 (+2304027us) service_pool.cc:224] Handling call
0709 12:13:44.408102 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.408716 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.408901 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.413622 23312 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:44.414557 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 344) took 2298 ms. Trace:
I20260709 12:13:44.415005 23312 rpcz_store.cc:276] 0709 12:13:42.115970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.410128 (+2294158us) service_pool.cc:224] Handling call
0709 12:13:44.412234 (+  2106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.414232 (+  1998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.414492 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.415409 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.422210 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:44.422969 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1459) took 1468 ms. Trace:
I20260709 12:13:44.423369 23462 rpcz_store.cc:276] 0709 12:13:42.954749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.954863 (+   114us) service_pool.cc:224] Handling call
0709 12:13:44.415223 (+1460360us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:44.422894 (+  7671us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9889536}
I20260709 12:13:42.903239 23394 rpcz_store.cc:276] 0709 12:13:40.683567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.683700 (+   133us) service_pool.cc:224] Handling call
0709 12:13:40.683944 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.684132 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:41.551484 (+867352us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.842082 (+1290598us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b18000ee4e0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:13:42.895657 (+ 53575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.895713 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:42.895755 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.895823 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.895931 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53552,"prepare.queue_time_us":1007991,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3353,"raft.queue_time_us":58966,"spinlock_wait_cycles":73210112,"thread_start_us":1003702,"threads_started":2,"wal-append.queue_time_us":283762}
I20260709 12:13:44.442555 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 566) took 2112 ms. Trace:
I20260709 12:13:44.443181 24016 rpcz_store.cc:276] 0709 12:13:42.330275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.460564 (+130289us) service_pool.cc:224] Handling call
0709 12:13:42.460964 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.480303 (+ 19339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.480337 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.480443 (+   106us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:42.480502 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.442216 (+1961714us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b540074dbd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:44.442270 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.442469 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43656448}
I20260709 12:13:44.446313 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.447319 23461 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
W20260709 12:13:44.463192 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2052) took 3781 ms (client timeout 1749 ms). Trace:
W20260709 12:13:44.465173 23396 rpcz_store.cc:269] 0709 12:13:40.681646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.681807 (+   161us) service_pool.cc:224] Handling call
0709 12:13:44.462566 (+3780759us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54002ea254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.463008 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3769806592}
W20260709 12:13:44.467376 23433 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.008s	sys 0.002s
W20260709 12:13:44.467809 23433 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.008s	sys 0.002s
I20260709 12:13:44.472890 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 345) took 2347 ms. Trace:
I20260709 12:13:44.473949 23312 rpcz_store.cc:276] 0709 12:13:42.125656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.416146 (+2290490us) service_pool.cc:224] Handling call
0709 12:13:44.416546 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.416794 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:44.425071 (+  8277us) log.cc:844] Serialized 14 byte log entry
0709 12:13:44.472457 (+ 47386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.472518 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.472578 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.472660 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.472827 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8199,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":104,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":54780,"threads_started":2,"wal-append.queue_time_us":47203}
I20260709 12:13:44.475589 23313 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:44.476284 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1480) took 2331 ms. Trace:
I20260709 12:13:44.476428 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 346) took 2333 ms. Trace:
I20260709 12:13:44.476835 23312 rpcz_store.cc:276] 0709 12:13:42.144502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.475504 (+2331002us) service_pool.cc:224] Handling call
0709 12:13:44.475777 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.475949 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.475968 (+    19us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.476044 (+    76us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.476085 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.476134 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.476233 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.477996 23313 rpcz_store.cc:276] 0709 12:13:42.142725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.474947 (+2332222us) service_pool.cc:224] Handling call
0709 12:13:44.475364 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.476135 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.476365 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.480408 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1484) took 2328 ms. Trace:
I20260709 12:13:44.480814 23313 rpcz_store.cc:276] 0709 12:13:42.151714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.479149 (+2327435us) service_pool.cc:224] Handling call
0709 12:13:44.479651 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.480002 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.480037 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.480090 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:44.480120 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.480189 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.480345 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.482756 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1469) took 1276 ms. Trace:
I20260709 12:13:44.490075 23469 rpcz_store.cc:276] 0709 12:13:43.206103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.206240 (+   137us) service_pool.cc:224] Handling call
0709 12:13:44.481883 (+1275643us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:44.482113 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.482421 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.482450 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.482491 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.482521 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.482580 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.482696 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":124,"spinlock_wait_cycles":24316288}
I20260709 12:13:44.482756 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1486) took 2330 ms. Trace:
I20260709 12:13:44.483423 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1481) took 2338 ms. Trace:
I20260709 12:13:44.493645 23312 rpcz_store.cc:276] 0709 12:13:42.144786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.478471 (+2333685us) service_pool.cc:224] Handling call
0709 12:13:44.479145 (+   674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.479542 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.479574 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.479635 (+    61us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:44.479670 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.479746 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.483363 (+  3617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:44.496201 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1496) took 2064 ms. Trace:
I20260709 12:13:44.496915 23312 rpcz_store.cc:276] 0709 12:13:42.431805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.495202 (+2063397us) service_pool.cc:224] Handling call
0709 12:13:44.495592 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.495799 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.495833 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.495882 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:44.495923 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.495986 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.496140 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:44.496418 23317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 348) took 2353 ms (client timeout 1749 ms). Trace:
W20260709 12:13:44.498761 23317 rpcz_store.cc:269] 0709 12:13:42.143384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.347547 (+204163us) service_pool.cc:224] Handling call
0709 12:13:44.496196 (+2148649us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b5400389d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.496321 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37332352}
I20260709 12:13:44.492657 23313 rpcz_store.cc:276] 0709 12:13:42.152363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.481866 (+2329503us) service_pool.cc:224] Handling call
0709 12:13:44.482144 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.482423 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.482452 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.482490 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:44.482523 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.482591 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.482696 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":415,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105}
I20260709 12:13:44.501677 23468 raft_consensus.cc:2368] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:44.505578 25124 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.506191 25124 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.508554 25124 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.514526 23470 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:44.517220 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1470) took 1260 ms. Trace:
I20260709 12:13:44.517628 23470 rpcz_store.cc:276] 0709 12:13:43.257113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.257268 (+   155us) service_pool.cc:224] Handling call
0709 12:13:44.516963 (+1259695us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002a0dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.517146 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2825654656}
I20260709 12:13:44.523540 24991 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:44.539229 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:44.540475 24995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e270393fc841ddac0d9170b0bc6554. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:13:44.540375 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:44.542008 25030 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:44.542701 25076 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:44.543447 25076 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:44.544435 25076 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.542439 25030 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.549291 25030 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:44.553746 25140 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.554454 25140 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.556980 25140 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.574527 24995 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:53940 (request call id 414) took 4300 ms. Trace:
I20260709 12:13:44.576227 24013 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:44.575204 24995 rpcz_store.cc:276] 0709 12:13:40.273929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.274118 (+   189us) service_pool.cc:224] Handling call
0709 12:13:42.038508 (+1764390us) ts_tablet_manager.cc:934] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
0709 12:13:44.539065 (+2500557us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:13:44.563860 (+ 24795us) tablet_metadata.cc:665] Metadata flushed
0709 12:13:44.563997 (+   137us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:13:44.573584 (+  9587us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:13:44.573956 (+   372us) tablet_replica.cc:206] Creating consensus instance
0709 12:13:44.574337 (+   381us) ts_tablet_manager.cc:989] Replica registered.
0709 12:13:44.574422 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":19933,"tablet-copy.queue_time_us":340740,"thread_start_us":956334,"threads_started":2}
I20260709 12:13:44.577018 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 564) took 2260 ms. Trace:
I20260709 12:13:44.577217 24995 tablet_copy_client.cc:806] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:13:44.577953 24995 tablet_copy_client.cc:670] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:13:44.577387 24013 rpcz_store.cc:276] 0709 12:13:42.316253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.344044 (+ 27791us) service_pool.cc:224] Handling call
0709 12:13:44.575785 (+2231741us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:44.576032 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.576729 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.576956 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8584064}
I20260709 12:13:44.583058 25050 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:44.588481 23314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1499) took 1529 ms (client timeout 1602 ms). Trace:
I20260709 12:13:44.592711 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:44.594786 23315 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:44.589015 23314 rpcz_store.cc:269] 0709 12:13:43.059588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.308040 (+248452us) service_pool.cc:224] Handling call
0709 12:13:44.588198 (+1280158us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400389854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.588410 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844780288}
I20260709 12:13:44.602178 25202 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:44.603392 25202 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:44.604386 25198 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:44.604770 25198 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:44.605978 25198 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.607342 25198 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.604393 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.610251 23320 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:44.611873 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.612618 23320 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:44.613962 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.617105 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.618897 23388 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:44.622473 23313 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:13:44.625064 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.625847 23313 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:44.626924 23465 raft_consensus.cc:2368] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
W20260709 12:13:44.628118 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1444) took 3790 ms (client timeout 1717 ms). Trace:
W20260709 12:13:44.628546 23465 rpcz_store.cc:269] 0709 12:13:40.837928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.845222 (+  7294us) service_pool.cc:224] Handling call
0709 12:13:43.363144 (+2517922us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:44.627900 (+1264756us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001063d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.628046 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2896807552}
I20260709 12:13:44.630234 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.635062 25198 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:44.645763 25053 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.646291 25053 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.647750 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.649581 25231 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:44.650992 25231 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:44.652423 25231 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.655391 25231 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:44.665728 25081 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:44.666280 25081 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:44.667352 25081 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.669063 25081 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.672456 25053 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.678900 25090 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:44.682924 23316 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.384s	user 0.015s	sys 0.018s
I20260709 12:13:44.690245 23316 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:44.691305 23316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1482) took 2546 ms (client timeout 1831 ms). Trace:
W20260709 12:13:44.693406 23316 rpcz_store.cc:269] 0709 12:13:42.145041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.373890 (+228849us) service_pool.cc:224] Handling call
0709 12:13:43.216725 (+842835us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b54001633a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a9677e18 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:44.691233 (+1474508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12034304}
I20260709 12:13:44.646261 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:44.701076 25231 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:44.701220 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55058.6us
I20260709 12:13:44.701522 25231 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.701905 25231 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:44.702631 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.703228 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:13:44.704859 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1454) took 1789 ms (client timeout 1554 ms). Trace:
W20260709 12:13:44.705250 23468 rpcz_store.cc:269] 0709 12:13:42.915780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.915912 (+   132us) service_pool.cc:224] Handling call
0709 12:13:44.704625 (+1788713us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540044b954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.704789 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3600522496}
I20260709 12:13:44.714215 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.714916 23316 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:44.709535 25231 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.719496 25231 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.719986 25030 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:44.725199 23319 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1495) took 2294 ms (client timeout 1693 ms). Trace:
I20260709 12:13:44.726991 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:44.727069 23319 rpcz_store.cc:269] 0709 12:13:42.430829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.949603 (+518774us) service_pool.cc:224] Handling call
0709 12:13:44.725016 (+1775413us) spinlock_profiling.cc:243] Waited 9.42 ms on lock 0x7b54003f98d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.725130 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21674240}
I20260709 12:13:44.727658 23316 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:44.725975 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:44.728597 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:44.730568 25030 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.731947 25231 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:44.734202 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.733672 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:44.742973 23317 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:44.699987 23313 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:44.731879 25202 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:44.745862 25202 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.746377 25202 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:44.752161 25232 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:44.758847 23317 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:44.744073 23316 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:44.744169 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:44.742693 25231 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.766103 25231 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:44.769761 25232 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:44.771020 25232 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:44.771365 25232 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.771713 25232 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:44.776873 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.779151 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.779981 23316 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:44.782923 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.778046 25083 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.784946 25083 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:44.786553 24950 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:13:44.787209 25083 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.787866 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.788590 23313 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:44.790524 25202 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.805269 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.806209 23316 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:44.806452 23317 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:44.807574 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:44.808710 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:44.792738 25202 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.826583 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:44.827518 23313 raft_consensus.cc:2376] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2. Re-sending same reply.
I20260709 12:13:44.824967 25083 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:44.829079 25083 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:44.830502 25083 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.830801 25232 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.824717 25186 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:44.834479 25198 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:44.834268 25186 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.834909 25198 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:13:44.837126 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 93017.8us
I20260709 12:13:44.839272 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:44.843480 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:44.844651 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:44.846621 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:44.847606 25099 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:44.848090 25099 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.848593 25099 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:44.849021 24991 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:44.849578 24991 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:44.850428 24991 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.853888 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:44.854863 23469 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:44.841668 24980 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.835790 25198 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.836869 25231 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.840257 25232 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.854907 25208 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:13:44.866788 25124 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:44.868196 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 563) took 2561 ms. Trace:
I20260709 12:13:44.885902 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:44.877804 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.877982 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.873996 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.887555 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.878633 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.865825 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:44.910362 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:44.857556 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:44.912082 23468 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:44.879554 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.889133 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.864516 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 341) took 2761 ms. Trace:
I20260709 12:13:44.913687 23315 rpcz_store.cc:276] 0709 12:13:42.103469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.406424 (+2302955us) service_pool.cc:224] Handling call
0709 12:13:44.594583 (+188159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.595240 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.864308 (+269068us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b5400389850. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:44.864452 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431996288}
I20260709 12:13:44.888504 24016 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:44.880887 24014 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:44.915436 23387 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:44.880402 24015 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:44.915725 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.920257 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.869292 23470 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:44.888681 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:44.922926 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.932600 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.934502 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.936697 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.936659 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.938862 23468 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:44.939005 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.941037 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.941205 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.942013 23393 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:13:44.942929 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.943948 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.946650 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.949534 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.951581 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:44.951767 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:44.952358 23393 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:44.952833 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.955230 23390 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 2: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:44.955103 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.955627 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.956040 24016 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:44.953368 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:44.875681 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:44.957739 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.877976 25237 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:44.861919 23480 tablet_copy_service.cc:342] P da1e2fd157894634bf92d6c82c06ec64: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ec4e3b5eed804f2a8c2a01e5d1ae5f53 received from {username='slave'} at 127.0.0.1:60318
I20260709 12:13:44.879652 25231 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:44.868672 25113 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.966101 25240 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:44.869262 25186 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:44.887667 25232 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:44.877568 25202 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:44.967013 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.967998 23396 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:44.879575 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.874261 25198 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.870448 25117 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:44.860520 25090 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:44.964428 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:44.987999 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.988318 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:44.971704 24980 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:44.965219 23315 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:44.972597 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.989526 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.989650 23390 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:44.990473 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:44.866492 25163 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:44.991551 25163 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.879880 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.974507 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.976686 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:44.888607 25124 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:44.884936 24012 rpcz_store.cc:276] 0709 12:13:42.293963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.302376 (+  8413us) service_pool.cc:224] Handling call
0709 12:13:44.713113 (+2410737us) spinlock_profiling.cc:243] Waited 4.96 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:44.713340 (+   227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.715386 (+  2046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.715433 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.715969 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.769084 (+ 53115us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:44.769130 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.769212 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.835761 (+ 66549us) raft_consensus.cc:1685] finished
0709 12:13:44.835794 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.854790 (+ 18996us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1376,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"spinlock_wait_cycles":11403008,"thread_start_us":50513,"threads_started":1,"wal-append.queue_time_us":109866}
I20260709 12:13:44.984491 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.987380 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.970129 23480 tablet_copy_service.cc:434] P da1e2fd157894634bf92d6c82c06ec64: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-ec4e3b5eed804f2a8c2a01e5d1ae5f53 on tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:44.968648 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:44.978971 25240 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.991194 23396 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:44.994309 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:44.978359 25159 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:44.996398 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:45.007155 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:45.015617 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.020220 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:45.026332 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.027431 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.027697 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.028129 23391 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:13:45.024242 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.028635 23320 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 2: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:45.038353 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:44.989152 25090 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:44.993033 25079 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:44.977905 25113 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.039378 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.049299 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:45.047542 23393 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:44.973976 25237 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.050765 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.057938 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.052124 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.054708 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.037739 25186 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:45.057906 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.022141 25232 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:45.024585 25208 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:45.066807 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.076128 23393 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:45.067530 23390 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:45.068598 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.069482 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.077577 23391 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:44.987644 25202 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:44.992851 25117 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.066702 24950 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:45.024490 25230 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:13:45.051406 25159 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:45.051802 25090 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.084318 24013 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:45.097196 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:45.099299 23391 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:45.099833 23468 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:45.100302 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.101037 23391 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:45.087199 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.104002 24991 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.108044 24014 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:13:45.107393 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.116932 23467 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:45.049690 25209 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:13:45.125650 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.092499 23312 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:45.118384 24015 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:45.118479 23396 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.079844 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.084280 24007 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:45.108672 25186 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:13:45.110365 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 182539us
I20260709 12:13:45.125149 25125 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.132835 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.148495 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.119362 25163 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.150537 23391 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:45.155694 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.157166 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.158682 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.152537 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.133977 25240 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.140339 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.161625 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.149206 25186 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:45.180279 25186 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:45.182381 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.183714 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.084746 25208 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.188416 25208 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.194132 25081 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:45.194783 25237 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.195998 25124 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:45.196833 25124 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:45.197368 25124 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:45.125295 24991 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.199892 23388 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:45.200978 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.201746 24015 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.184742 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.207991 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.209991 23390 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:45.214706 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 144468us
I20260709 12:13:45.215605 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.199151 24995 tablet_copy_client.cc:538] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:13:45.221233 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.223028 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:45.223665 23393 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:45.225021 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 72602.9us
W20260709 12:13:45.225871 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 171280us
I20260709 12:13:45.160744 25125 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.228246 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.208492 25081 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.203204 24008 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:45.198329 25124 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:45.208184 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.235266 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:45.222245 25239 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:45.217478 25092 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:45.220795 25079 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:45.226464 24009 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:45.220732 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 550) took 5187 ms (client timeout 2013 ms). Trace:
I20260709 12:13:45.220389 25232 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:45.253854 23390 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:45.258534 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.268479 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.259366 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.259538 24015 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:45.264072 23467 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
W20260709 12:13:45.296159 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61125us
I20260709 12:13:45.223301 25208 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:13:45.310248 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:45.308806 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83957.5us
I20260709 12:13:45.254604 25239 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:45.221326 25159 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.317027 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.317655 24015 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:45.313033 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:45.245183 25081 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.309625 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:45.258963 24011 rpcz_store.cc:269] 0709 12:13:40.025171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.029657 (+  4486us) service_pool.cc:224] Handling call
0709 12:13:40.864034 (+834377us) spinlock_profiling.cc:243] Waited 8.91 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:45.185421 (+4321387us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54006702d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:45.207426 (+ 22005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247438464}
I20260709 12:13:45.329938 25232 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.312314 25092 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:45.331997 23390 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:13:45.376226 23390 pending_rounds.cc:85] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 4
I20260709 12:13:45.379021 23390 pending_rounds.cc:107] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:13:45.379370 23390 raft_consensus.cc:2889] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:45.318301 25117 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.325718 25113 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.327162 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.317781 25239 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:45.412842 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 142250us
I20260709 12:13:45.443989 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.378293 25092 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:45.310770 25250 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:45.316813 25202 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.375155 25254 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:45.450419 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.377154 24991 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:45.438141 25248 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:13:45.412252 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:13:45.438127 25251 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:13:45.466055 23396 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:45.450806 25092 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.469619 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:45.408290 25083 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:45.420738 25209 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:45.438714 25240 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:45.479213 24016 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:45.481372 25261 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:13:45.421132 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 108178us
W20260709 12:13:45.482372 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 176970us
I20260709 12:13:45.443800 25260 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:13:45.497721 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 257301us
I20260709 12:13:45.499429 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.504112 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:45.467420 25254 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:45.468436 24008 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:13:45.515779 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.513940 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.510569 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.473917 25100 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:45.504727 25250 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:45.525504 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.412910 25079 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.506940 25088 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:45.504724 25240 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:45.508469 25249 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.521917 24011 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:45.511890 25209 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:45.532717 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.533542 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.534323 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.535380 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:45.530844 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.546455 24015 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:45.546563 23469 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:45.471472 24980 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:13:45.538209 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.521945 25125 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.549539 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:45.556025 23468 raft_consensus.cc:2368] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:45.528189 25250 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:45.550752 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:45.523027 24012 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:45.509327 25186 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.550160 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.556650 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.529551 25088 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:45.539333 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.560603 23463 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:45.548398 25117 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:45.550873 25260 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:45.551844 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.548645 25261 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.549743 25258 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:45.574731 25050 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:45.569818 23461 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:45.586758 25258 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.587800 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.577626 25240 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:45.577327 25099 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.728s	user 0.012s	sys 0.000s
I20260709 12:13:45.582012 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.568444 25260 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:45.582051 24011 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:45.551298 25266 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:13:45.555114 25265 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:13:45.560913 25250 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.583699 25186 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.562841 25254 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.585723 25081 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.538192 25264 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 12:13:45.563939 25088 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:45.565850 25117 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.597512 25261 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.541009 25249 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.570104 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.600467 25265 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:45.598964 25240 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:45.624044 25265 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:13:45.611711 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:45.625852 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.599857 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.599226 23469 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:45.599077 24010 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:45.652118 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:45.600906 24037 debug-util.cc:398] Leaking SignalData structure 0x7b0800272720 after lost signal to thread 22110
I20260709 12:13:45.634105 25050 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.654798 25088 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.598723 25099 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.609679 25117 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.603001 25254 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:45.620229 24011 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:45.607620 25113 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:45.605511 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.682798 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.611277 25261 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.606263 25258 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:45.599380 25260 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.599992 25266 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:45.611408 25090 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.613734 23464 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:45.784089 23394 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:45.819545 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.612999 25249 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.820187 23390 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:45.775220 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:13:45.812435 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 228311us
I20260709 12:13:45.770743 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.599303 25209 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.830883 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.821280 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:45.835143 23468 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:45.862982 23392 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:45.862671 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.757942 25177 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:45.859826 24016 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:45.598419 25092 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.864791 23391 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 2: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:13:45.872300 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 51095.8us
I20260709 12:13:45.814957 25113 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:45.814924 25266 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:45.861033 25088 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.859812 25099 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.867380 25250 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.711980 25050 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:45.875293 25124 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:45.887341 25124 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.783319 25265 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.892606 25266 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.896130 25266 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.899308 25260 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.901038 25260 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.901861 25250 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:45.940779 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 402441us
I20260709 12:13:45.947299 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:45.948741 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:45.956211 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 180953us
I20260709 12:13:45.878717 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.877345 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.961932 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.959717 23464 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:45.960034 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:45.964984 23468 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:45.966566 23396 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:13:45.967183 23396 pending_rounds.cc:85] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 5
I20260709 12:13:45.967267 24995 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:13:45.967521 23396 pending_rounds.cc:107] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:13:45.964079 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:45.821581 23461 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 1.
I20260709 12:13:45.968003 23396 raft_consensus.cc:2889] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:45.974913 25050 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.975078 23465 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:45.977391 25050 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:45.959940 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:45.980832 23390 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:45.969890 23468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:45.981846 23468 raft_consensus.cc:606] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:45.994354 23391 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:45.994366 23468 raft_consensus.cc:614] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:45.998205 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:45.999074 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1273) took 1030 ms. Trace:
I20260709 12:13:45.999408 23387 rpcz_store.cc:276] 0709 12:13:44.968695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.997714 (+1029019us) service_pool.cc:224] Handling call
0709 12:13:45.999012 (+  1298us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:46.007829 24037 debug-util.cc:398] Leaking SignalData structure 0x7b080027cde0 after lost signal to thread 23253
I20260709 12:13:46.010732 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2089) took 1124 ms. Trace:
I20260709 12:13:46.011207 23388 rpcz_store.cc:276] 0709 12:13:44.886538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.888471 (+  1933us) service_pool.cc:224] Handling call
0709 12:13:44.888735 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.200276 (+311541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.010109 (+809833us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:46.010482 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715175808}
I20260709 12:13:46.015453 23466 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:45.982244 25050 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:45.986938 25090 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:45.987776 25092 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:45.988550 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:45.990226 25099 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:45.991293 25113 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:45.993748 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 531) took 1111 ms. Trace:
I20260709 12:13:46.017602 25124 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.020251 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2088) took 1133 ms. Trace:
W20260709 12:13:46.022600 24037 debug-util.cc:398] Leaking SignalData structure 0x7b0800387260 after lost signal to thread 23345
W20260709 12:13:46.027096 24037 debug-util.cc:398] Leaking SignalData structure 0x7b08002ccba0 after lost signal to thread 23346
I20260709 12:13:46.029979 25261 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:46.032155 25261 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:46.032668 25261 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.033921 25239 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.034960 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.037400 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.037832 24013 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:46.038667 23466 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:46.039286 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 581) took 2340 ms. Trace:
W20260709 12:13:46.039832 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1479) took 2196 ms (client timeout 1640 ms). Trace:
W20260709 12:13:46.040529 23466 rpcz_store.cc:269] 0709 12:13:43.843697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.933969 (+ 90272us) service_pool.cc:224] Handling call
0709 12:13:46.039591 (+2105622us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.039734 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4781893504}
I20260709 12:13:46.041038 23314 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:46.024930 23389 rpcz_store.cc:276] 0709 12:13:44.886254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.886399 (+   145us) service_pool.cc:224] Handling call
0709 12:13:44.886678 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.992951 (+106273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.992980 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.993020 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:44.993050 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.018345 (+1025295us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b540036d654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:46.018405 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.019570 (+  1165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244014848}
I20260709 12:13:46.037127 25239 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:46.042469 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 93943us
I20260709 12:13:46.019670 25050 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.046115 25244 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.734s	user 0.007s	sys 0.009s
W20260709 12:13:46.046659 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.046401 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.047247 25125 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:46.049002 25099 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:46.051098 25113 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:46.043113 24037 debug-util.cc:398] Leaking SignalData structure 0x7b080038d060 after lost signal to thread 23420
I20260709 12:13:46.053017 25261 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.053452 25258 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:46.054239 23434 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.082s (SENT)
W20260709 12:13:46.046609 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 58394.5us
W20260709 12:13:46.059363 24037 debug-util.cc:398] Leaking SignalData structure 0x7b080038ca80 after lost signal to thread 23421
I20260709 12:13:46.056874 25050 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.047283 25266 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:46.039714 24009 rpcz_store.cc:276] 0709 12:13:43.698839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.699005 (+   166us) service_pool.cc:224] Handling call
0709 12:13:45.206927 (+1507922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.258838 (+ 51911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.038879 (+780041us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54006702d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.039118 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3461085952}
W20260709 12:13:46.073022 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56888us
I20260709 12:13:46.074805 25254 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.024559 24015 rpcz_store.cc:276] 0709 12:13:44.881768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.551395 (+669627us) service_pool.cc:224] Handling call
0709 12:13:45.551821 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.552091 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:45.778138 (+226047us) log.cc:844] Serialized 14 byte log entry
0709 12:13:45.992931 (+214793us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:45.993049 (+   118us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:45.993163 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.993284 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.993551 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":306162,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":27956,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":324,"thread_start_us":388951,"threads_started":2,"wal-append.queue_time_us":88290}
I20260709 12:13:46.070437 25250 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:46.079535 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.080749 23315 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:46.071981 25125 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.069087 25266 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:46.087215 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:46.090377 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.078614 25250 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.093972 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:46.092571 25099 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.094625 25250 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.095410 24015 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:13:46.096705 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.096297 25250 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:46.097515 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.092621 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:46.098585 24007 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:46.085554 25266 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:46.100697 24037 debug-util.cc:398] Leaking SignalData structure 0x7b0800329ea0 after lost signal to thread 23494
I20260709 12:13:46.105010 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.083297 25125 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:46.107448 24037 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3260 after lost signal to thread 23495
W20260709 12:13:46.117808 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.115631 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.120882 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 527) took 1240 ms. Trace:
I20260709 12:13:46.112028 23313 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:46.121313 24014 rpcz_store.cc:276] 0709 12:13:44.880626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.057603 (+176977us) service_pool.cc:224] Handling call
0709 12:13:46.120469 (+1062866us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54007c01d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.120667 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108308736}
I20260709 12:13:46.132138 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.132174 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:46.133164 23320 raft_consensus.cc:606] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.135130 23315 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:46.137739 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:46.137785 25148 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:46.138882 23315 raft_consensus.cc:606] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:46.141647 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.111855 25254 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.144073 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:46.150345 25076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.563s	user 0.012s	sys 0.003s
W20260709 12:13:46.145216 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 58359.2us
W20260709 12:13:46.152009 25076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.565s	user 0.013s	sys 0.003s
I20260709 12:13:46.152575 25254 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:46.153355 25232 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.155504 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.163753 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 582) took 1015 ms. Trace:
I20260709 12:13:46.164224 24013 rpcz_store.cc:276] 0709 12:13:45.148559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.610395 (+461836us) service_pool.cc:224] Handling call
0709 12:13:45.610671 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.038313 (+427642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.163405 (+125092us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54003741d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:46.163655 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981483264}
W20260709 12:13:46.166359 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 111465us
I20260709 12:13:46.175539 24012 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:46.181200 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.182232 25090 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.183225 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.183746 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.184846 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.185165 25090 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:46.185684 23463 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:46.186251 25245 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.186949 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.188019 25090 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:46.187037 23216 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:46.188791 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 533) took 1306 ms. Trace:
W20260709 12:13:46.161168 25274 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 12:13:46.190629 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2095) took 1272 ms. Trace:
I20260709 12:13:46.190416 25090 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:46.191103 23394 rpcz_store.cc:276] 0709 12:13:44.917959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.919961 (+  2002us) service_pool.cc:224] Handling call
0709 12:13:46.190130 (+1270169us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54003449d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.190378 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1984408960}
I20260709 12:13:46.193362 23218 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 313) took 6288 ms. Trace:
I20260709 12:13:46.195201 25163 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:46.195737 25163 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:46.197855 25279 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.380s	user 0.000s	sys 0.004s
I20260709 12:13:46.198033 23395 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:46.197351 25163 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.203897 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.206468 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1261) took 1274 ms. Trace:
I20260709 12:13:46.206939 23395 rpcz_store.cc:276] 0709 12:13:44.932502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.932673 (+   171us) service_pool.cc:224] Handling call
0709 12:13:46.197653 (+1264980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.198529 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.206244 (+  7715us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003449d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.206382 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2908079872}
I20260709 12:13:46.208794 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.210177 25089 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:46.210261 23434 leader_election.cc:336] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.242s (SENT)
I20260709 12:13:46.213795 25114 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:46.216210 25177 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:46.216977 25177 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.189240 24012 rpcz_store.cc:276] 0709 12:13:44.882390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.572939 (+690549us) service_pool.cc:224] Handling call
0709 12:13:46.174660 (+601721us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:46.174999 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.176061 (+  1062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.188719 (+ 12658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26215808}
I20260709 12:13:46.221022 23395 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:46.221643 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.223088 25239 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.224359 25265 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.228055 25083 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:46.228780 23465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:46.229405 25083 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:46.229943 25083 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:46.231209 25083 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.232846 25083 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.229370 23465 raft_consensus.cc:606] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.238436 25261 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.193799 23218 rpcz_store.cc:276] 0709 12:13:39.904729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.904898 (+   169us) service_pool.cc:224] Handling call
0709 12:13:46.193064 (+6288166us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:40.444762 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:40.568346 (+123584us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:40.572386 (+  4040us) write_op.cc:435] Acquired schema lock
0709 12:13:40.573930 (+  1544us) tablet.cc:662] Decoding operations
0709 12:13:40.584526 (+ 10596us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:40.584609 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:40.584656 (+    47us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:40.584906 (+   250us) tablet.cc:701] Row locks acquired
0709 12:13:40.584931 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:13:40.585123 (+   192us) write_op.cc:270] Start()
0709 12:13:40.585269 (+   146us) write_op.cc:276] Timestamp: P: 1783599220585065 usec, L: 0
0709 12:13:40.585300 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:13:40.586462 (+  1162us) log.cc:844] Serialized 539 byte log entry
0709 12:13:42.668180 (+2081718us) op_driver.cc:464] REPLICATION: finished
0709 12:13:43.216532 (+548352us) write_op.cc:301] APPLY: starting
0709 12:13:43.219851 (+  3319us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:43.256267 (+ 36416us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:43.270682 (+ 14415us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:43.271044 (+   362us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:43.271273 (+   229us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:43.271355 (+    82us) write_op.cc:312] APPLY: finished
0709 12:13:43.304397 (+ 33042us) log.cc:844] Serialized 23 byte log entry
0709 12:13:43.309363 (+  4966us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:43.310893 (+  1530us) write_op.cc:454] Released schema lock
0709 12:13:43.323849 (+ 12956us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:43.409288 (+ 85439us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:43.501270 (+ 91982us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8538,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":491382,"apply.run_cpu_time_us":3296,"apply.run_wall_time_us":182031,"num_ops":1,"prepare.queue_time_us":294161,"prepare.run_cpu_time_us":29810,"prepare.run_wall_time_us":976379,"raft.queue_time_us":610688,"raft.run_cpu_time_us":44346,"raft.run_wall_time_us":390524,"replication_time_us":2082691,"thread_start_us":1166410,"threads_started":4,"wal-append.queue_time_us":975140,"wal-append.run_cpu_time_us":9055,"wal-append.run_wall_time_us":2748732}]]}
I20260709 12:13:46.241760 25249 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:13:46.236394 24037 debug-util.cc:398] Leaking SignalData structure 0x7b08003871e0 after lost signal to thread 24040
I20260709 12:13:46.241756 23315 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:46.246217 24037 debug-util.cc:398] Leaking SignalData structure 0x7b08003871a0 after lost signal to thread 24041
I20260709 12:13:46.253960 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.255329 25186 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:46.255445 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.257845 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.258455 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.262651 23463 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:46.263007 25163 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.264400 23319 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:46.243149 25249 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:13:46.266398 25249 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:46.266809 25249 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.267036 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2104) took 1283 ms. Trace:
I20260709 12:13:46.267138 23465 raft_consensus.cc:2376] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2. Re-sending same reply.
W20260709 12:13:46.268326 24037 debug-util.cc:398] Leaking SignalData structure 0x7b0800387340 after lost signal to thread 25048
I20260709 12:13:46.268040 25249 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.269796 25249 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.257491 25186 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.271330 25186 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.272787 25186 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.270167 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 434) took 1388 ms. Trace:
I20260709 12:13:46.267649 23393 rpcz_store.cc:276] 0709 12:13:44.983113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.110640 (+127527us) service_pool.cc:224] Handling call
0709 12:13:45.110940 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.224028 (+113088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.266593 (+1042565us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540043c1d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:46.266863 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253258112}
I20260709 12:13:46.297326 23312 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:46.303488 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 82069.1us
I20260709 12:13:46.301628 25209 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:13:46.307053 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.307158 23312 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:46.307650 25209 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:46.307922 23312 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.308387 23312 raft_consensus.cc:614] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:46.308357 25209 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.309648 25257 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:46.310048 25257 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.311564 25257 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.312994 25237 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:46.314222 25198 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:46.315572 25239 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:46.315970 25239 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:46.316144 25050 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.317107 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:46.313160 25257 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.314298 25237 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:46.326046 23318 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:46.328555 25030 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:13:46.329741 25030 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:46.290426 25124 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.333889 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:13:46.336442 25117 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.009s
I20260709 12:13:46.341068 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:46.342363 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:46.330090 25030 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:46.341147 23395 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:46.273662 24008 rpcz_store.cc:276] 0709 12:13:44.881765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.552935 (+671170us) service_pool.cc:224] Handling call
0709 12:13:46.130607 (+577672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.130998 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.131023 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:46.131087 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:46.131123 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.131198 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.269890 (+138692us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54007c01d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:46.270007 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279024384}
I20260709 12:13:46.351879 23219 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 339) took 5517 ms. Trace:
I20260709 12:13:46.352414 23219 rpcz_store.cc:276] 0709 12:13:40.834772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.834924 (+   152us) service_pool.cc:224] Handling call
0709 12:13:42.265621 (+1430697us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:46.351633 (+4086012us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1020071,"reported_tablets":2,"spinlock_wait_cycles":1198906112}
I20260709 12:13:46.356539 23395 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:46.357180 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:46.357748 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.358031 23392 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:46.359205 25117 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.317005 25239 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.361217 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.362217 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.366564 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.372673 23469 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
W20260709 12:13:46.379307 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 51622us
I20260709 12:13:46.379382 25124 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.380559 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.381837 25053 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:46.382862 23465 raft_consensus.cc:2376] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5. Re-sending same reply.
I20260709 12:13:46.383124 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:46.383827 24008 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:46.386061 24008 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:13:46.387671 25050 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.374742 25148 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:46.394625 25148 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:46.395826 25148 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.397032 25148 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.407310 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.416461 23393 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:13:46.418984 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74583.4us
I20260709 12:13:46.420133 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.420825 23461 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.424340 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:46.428937 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:46.433720 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53348.4us
I20260709 12:13:46.430812 23463 raft_consensus.cc:2376] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2. Re-sending same reply.
I20260709 12:13:46.433061 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.447151 25264 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.450659 24012 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:46.452440 23469 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:46.453269 25117 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.456697 25239 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.453336 23466 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:46.462519 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:46.462514 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.473013 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:46.372951 25030 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.488858 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:46.490710 23467 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:46.490866 23466 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:46.502517 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.485347 25175 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:46.502980 23465 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:46.483644 24008 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
W20260709 12:13:46.489558 23287 connection.cc:256] RPC call timeout handler was delayed by -1.2659932169999593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 12:13:46.512197 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.504945 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:13:46.507390 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1813932170000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 12:13:46.513095 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1390932169999815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 12:13:46.513358 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0112932170000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:13:46.515518 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0112932170000022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:13:46.515745 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0031932169999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 12:13:46.515882 23465 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
W20260709 12:13:46.516021 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0031932169999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260709 12:13:46.517421 23392 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:46.472638 25266 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.518576 23395 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:46.519282 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.520756 23467 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.521257 25266 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.526065 25030 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.527261 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.529913 25099 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.530920 25209 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.530794 25177 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.531252 25099 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:46.532858 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.534154 23362 leader_election.cc:400] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4.
I20260709 12:13:46.534368 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.535041 25232 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:46.535902 25232 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.536245 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.537307 25148 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.538789 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.537099 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:46.541350 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.545900 23466 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:46.533154 25177 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.547245 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.550006 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:46.554324 23392 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:46.555791 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.559289 23462 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:13:46.560742 23462 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:46.561621 25076 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:46.562206 25076 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:46.567354 23467 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:46.570324 23394 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.552588 25090 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:46.580055 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.577798 23318 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:46.549991 25186 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:13:46.582758 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.579649 25177 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.563181 25076 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:46.598616 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.598873 25090 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.599440 25090 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:46.579344 25237 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:13:46.579541 25261 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 12:13:46.609656 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.609864 23462 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:46.589911 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:13:46.621810 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 72293.5us
W20260709 12:13:46.631731 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45254.9us
I20260709 12:13:46.604673 25249 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:13:46.644292 25303 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:13:46.647414 23312 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:46.630179 25258 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:46.650779 25258 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.655575 23464 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:46.617983 25083 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:13:46.658067 25249 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.659222 25249 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.660650 25249 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.670457 23461 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:46.671305 23465 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:46.672161 24995 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:13:46.674324 24995 tablet_bootstrap.cc:492] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:13:46.675776 24995 ts_tablet_manager.cc:1403] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.710s	user 0.039s	sys 0.012s
W20260709 12:13:46.676707 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.678651 24014 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:46.653160 25258 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:46.682667 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.687430 23394 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:46.631510 25163 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:46.693768 25148 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:46.698561 25302 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:13:46.701328 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:46.709395 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.713860 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.701457 25163 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.715798 23392 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:46.720314 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:46.698117 25232 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:13:46.722884 25250 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.722126 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.717888 25163 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.725294 25257 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:46.727752 25257 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.728801 25257 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.726357 25261 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:46.731758 25261 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:46.732159 25261 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:46.737772 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.746874 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:46.748047 25089 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:46.748504 25089 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.748869 25089 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.750770 25163 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.753494 23395 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:46.753775 25163 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.754482 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.762763 25090 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.765097 25090 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.770352 24013 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:46.765597 25186 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:13:46.777259 25265 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:46.765215 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.792415 23464 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:46.793792 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.795174 25250 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:46.796326 24899 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:46.798345 25310 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:13:46.799228 24899 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:46.799686 24899 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.806607 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.807572 25262 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:46.806736 25250 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.808075 25262 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.808504 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:46.809077 25262 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.810340 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:46.811877 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 84820.3us
I20260709 12:13:46.817675 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.821463 25259 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:46.822036 25259 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.830036 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:46.832273 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.832026 25053 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:13:46.837651 25259 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.842176 23461 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:46.842913 25090 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:46.843492 25090 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.843978 25090 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.846443 25209 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:46.846988 25209 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:46.862361 23465 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:46.866583 23470 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:46.868254 25053 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:46.870100 25089 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.870312 24995 raft_consensus.cc:359] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:46.871163 24995 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
W20260709 12:13:46.866745 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53975.4us
I20260709 12:13:46.860690 25261 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:46.880434 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 43541.7us
I20260709 12:13:46.881335 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.882349 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.874444 25053 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:46.848429 25209 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.897945 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.889305 25114 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:46.900024 25114 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.900539 25114 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.901878 25053 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.905293 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:46.887497 24899 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.871860 24995 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:46.911226 25125 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:46.914940 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:46.911238 24899 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.917675 25163 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:46.918224 25209 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:46.919978 25163 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.920429 25163 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.897465 25089 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.927775 25125 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.931408 25261 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.931351 25125 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.932186 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.933419 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.934742 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.934844 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:46.934929 25076 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:46.935734 25076 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:46.936862 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:46.933507 25245 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:46.937646 25245 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:46.938066 25245 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:46.938540 25250 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:46.937815 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:46.939069 25114 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.940189 25050 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:46.940685 25050 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:46.941821 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.941691 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.940694 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.943013 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.943671 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.944823 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.945282 25148 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.935858 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.945423 25114 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:46.941617 25050 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.949002 24995 ts_tablet_manager.cc:1434] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.273s	user 0.008s	sys 0.000s
I20260709 12:13:46.950131 25089 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.951327 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-87e270393fc841ddac0d9170b0bc6554 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:13:46.951231 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.951965 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-87e270393fc841ddac0d9170b0bc6554 on tablet 87e270393fc841ddac0d9170b0bc6554 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:46.952595 25232 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:46.952756 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:46.970283 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.972533 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.974397 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.975281 23313 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:46.978102 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.978550 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.980661 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.981341 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.981756 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.982568 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.982987 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.983529 25089 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.984062 25089 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:46.978856 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.975328 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:46.979835 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.986234 25089 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:46.986763 25089 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:46.979996 23316 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:46.987251 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.988310 25114 raft_consensus.cc:2764] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:13:46.988926 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.989816 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:46.990541 25089 raft_consensus.cc:2764] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:13:46.992772 25089 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:46.994885 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.995500 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.995729 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.996177 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.996605 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.996938 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:46.997195 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.997985 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.998318 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.998858 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:46.999606 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:46.999773 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.994838 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.002751 25089 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:47.002900 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.003767 25089 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:47.005159 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.006251 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.005882 25089 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.006942 25163 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.008116 23216 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 324) took 6785 ms. Trace:
I20260709 12:13:47.009550 25163 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.000299 25090 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:46.939101 25250 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:47.009164 23217 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:47.025645 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.025906 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.015642 25232 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:13:47.020802 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.037148 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.009639 23216 rpcz_store.cc:276] 0709 12:13:40.222361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.222473 (+   112us) service_pool.cc:224] Handling call
0709 12:13:47.008055 (+6785582us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:46.421560 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:46.458079 (+ 36519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:46.458138 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:13:46.458153 (+    15us) tablet.cc:662] Decoding operations
0709 12:13:46.458556 (+   403us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:46.458637 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:46.458673 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:46.458978 (+   305us) tablet.cc:701] Row locks acquired
0709 12:13:46.459011 (+    33us) write_op.cc:260] PREPARE: finished
0709 12:13:46.459199 (+   188us) write_op.cc:270] Start()
0709 12:13:46.459343 (+   144us) write_op.cc:276] Timestamp: P: 1783599226459137 usec, L: 0
0709 12:13:46.459382 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:13:46.460097 (+   715us) log.cc:844] Serialized 539 byte log entry
0709 12:13:46.827289 (+367192us) op_driver.cc:464] REPLICATION: finished
0709 12:13:46.951420 (+124131us) write_op.cc:301] APPLY: starting
0709 12:13:46.951575 (+   155us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:46.951952 (+   377us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:46.964887 (+ 12935us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:47.001415 (+ 36528us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:47.001744 (+   329us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:47.001830 (+    86us) write_op.cc:312] APPLY: finished
0709 12:13:47.002935 (+  1105us) log.cc:844] Serialized 23 byte log entry
0709 12:13:47.003754 (+   819us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:47.003843 (+    89us) write_op.cc:454] Released schema lock
0709 12:13:47.003914 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:47.004416 (+   502us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:47.004453 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1580089,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":123699,"apply.run_cpu_time_us":2717,"apply.run_wall_time_us":52989,"num_ops":1,"prepare.queue_time_us":129513,"prepare.run_cpu_time_us":12790,"prepare.run_wall_time_us":297263,"raft.queue_time_us":28943,"raft.run_cpu_time_us":34381,"raft.run_wall_time_us":52875,"replication_time_us":367710,"thread_start_us":375893,"threads_started":4,"wal-append.queue_time_us":335730}]]}
I20260709 12:13:47.037869 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.041126 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.041994 25090 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.043708 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.046315 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.047662 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.048053 24009 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEARNER]: Advancing to term 4
I20260709 12:13:47.048393 24012 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:47.047987 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.049811 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.020566 25239 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:13:47.050771 25114 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:47.051309 25114 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:47.052244 25114 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.053045 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.055166 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.055588 25177 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:47.020977 25053 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:13:47.057396 25250 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:47.060870 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.062633 23217 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 216) took 4807 ms. Trace:
I20260709 12:13:47.063656 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.063751 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.063963 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.036474 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.064815 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.064981 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.064949 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.056133 25177 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:47.077373 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.080868 25090 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.081408 25090 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.082288 23468 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:47.083420 25090 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:47.084964 24014 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:47.085909 25209 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.086453 25209 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.087411 25245 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.088663 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.089618 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.090548 25177 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.090919 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.091604 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.092227 23468 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:13:47.092800 23313 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:47.093257 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:47.118707 23392 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:47.118839 23320 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:47.102293 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:47.110502 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:47.112017 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.113826 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.119650 23466 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:47.130770 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.132575 23463 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:47.135564 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.128753 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.138821 23470 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:47.131791 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.143446 23469 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.144057 23462 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:47.141788 25050 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:47.144634 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.150173 25050 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.063005 23217 rpcz_store.cc:276] 0709 12:13:42.255538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.255715 (+   177us) service_pool.cc:224] Handling call
0709 12:13:47.062575 (+4806860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:47.053961 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:47.054608 (+   647us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:47.054644 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:13:47.054665 (+    21us) tablet.cc:662] Decoding operations
0709 12:13:47.055046 (+   381us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:47.055095 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:47.055126 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:13:47.055303 (+   177us) tablet.cc:701] Row locks acquired
0709 12:13:47.055322 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:13:47.055461 (+   139us) write_op.cc:270] Start()
0709 12:13:47.055573 (+   112us) write_op.cc:276] Timestamp: P: 1783599227055425 usec, L: 0
0709 12:13:47.055597 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:13:47.056198 (+   601us) log.cc:844] Serialized 539 byte log entry
0709 12:13:47.058093 (+  1895us) op_driver.cc:464] REPLICATION: finished
0709 12:13:47.059050 (+   957us) write_op.cc:301] APPLY: starting
0709 12:13:47.059129 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:47.059228 (+    99us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:47.059319 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:47.059568 (+   249us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:47.059717 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:13:47.059767 (+    50us) write_op.cc:312] APPLY: finished
0709 12:13:47.059942 (+   175us) log.cc:844] Serialized 23 byte log entry
0709 12:13:47.060099 (+   157us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:47.060179 (+    80us) write_op.cc:454] Released schema lock
0709 12:13:47.060212 (+    33us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:47.060554 (+   342us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:47.060579 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":731,"apply.run_cpu_time_us":1458,"apply.run_wall_time_us":1458,"num_ops":1,"prepare.queue_time_us":895,"prepare.run_cpu_time_us":2531,"prepare.run_wall_time_us":2531,"replication_time_us":2370}]]}
I20260709 12:13:47.150794 23464 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.152506 25050 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:47.152936 23394 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:47.153599 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.154147 23396 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:13:47.154261 24016 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:47.155119 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.154177 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.145129 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:47.159512 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.172092 23465 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:47.143404 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.178474 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:47.181342 23466 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:47.173877 23392 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:47.142800 25239 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:13:47.206182 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:13:47.207074 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46282.9us
W20260709 12:13:47.207526 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53461.9us
I20260709 12:13:47.166471 25209 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:47.166592 25076 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:47.184105 25245 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.173534 25089 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:47.220701 25257 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:47.163822 25198 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:13:47.205468 25261 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:47.231443 25076 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.233657 25076 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.228822 25114 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:47.235680 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.212347 25030 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.238663 23395 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:47.219826 25089 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.241370 25089 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.242183 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.243359 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.244223 25209 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:47.244841 25209 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:47.245887 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.246515 25030 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.247920 24009 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:47.248885 25030 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.225755 25239 consensus_queue.cc:1243] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:13:47.249905 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.250844 23312 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:47.234889 25198 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:47.252830 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.253278 25198 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:47.257911 25177 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:47.258287 25114 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:47.258787 25114 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:47.259650 25114 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.261035 25114 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:47.262326 25239 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.262843 25239 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.263453 23464 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:47.264863 25239 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.258347 25177 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.265790 25177 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.267184 23469 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:47.269877 25209 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:47.269806 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.275565 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.276518 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.277704 25050 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:47.278164 23315 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:47.278692 23315 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:47.279096 25336 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:47.279527 25336 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:47.279723 23315 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.280412 25336 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.281049 23315 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.278167 25050 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.282132 25336 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.278970 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:47.289889 23320 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:47.287496 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.254452 25198 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.298072 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.294111 25264 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:47.302475 25250 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:47.302798 25264 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:47.303758 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:47.304014 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:47.304395 23392 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:47.304924 25250 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:47.305105 25264 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.305814 25090 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:13:47.311471 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:47.312058 25163 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:47.312353 24899 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:47.310864 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:47.296573 25050 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.314203 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:47.302173 23395 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:47.328449 24899 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:47.328819 24899 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.329171 24899 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:47.316861 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.301357 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:47.335919 23393 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:47.337173 23392 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:47.337606 23469 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.338006 24011 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:47.340500 25089 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.341487 25265 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:47.350509 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.340500 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.315009 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.310950 25198 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.359613 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:47.352180 23466 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:47.362833 23465 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:47.364931 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.373142 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:47.374862 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 78661.4us
I20260709 12:13:47.377805 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:47.378422 23463 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.357179 23219 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:47.381417 23219 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 5 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:47.385310 23469 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:47.384570 23219 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:47.386461 23468 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.383587 23464 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:47.386955 23219 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:47.400125 23219 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:47.401798 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.393533 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:47.412014 25163 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:47.413877 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.415561 25050 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.392619 24899 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.418136 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.420214 25262 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:47.420634 25262 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.420924 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:47.372874 25076 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:47.465813 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.421146 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.439361 24899 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:47.421589 23389 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.475497 23315 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:47.380510 25232 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:13:47.476445 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 104003us
I20260709 12:13:47.367175 25261 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:13:47.421044 25262 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.445415 23468 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:13:47.486249 23434 leader_election.cc:336] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.512s (SENT)
I20260709 12:13:47.488186 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:47.483764 23463 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:47.404878 23219 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 2 to 3, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:47.491155 25089 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:47.378685 25177 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.486007 25265 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:47.492776 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.493669 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.481395 25232 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:47.497578 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:47.500650 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 79583.1us
I20260709 12:13:47.517503 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:47.533288 25239 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.534080 25239 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.536729 25239 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.499161 23219 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 5, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:47.502143 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:47.484951 25250 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:47.505519 23317 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:47.425691 23312 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:47.509557 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.550027 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.550738 25053 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:47.551412 25053 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.551939 25053 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.539062 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:47.554546 25245 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:47.555991 25237 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:47.556551 25237 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:47.563840 23219 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:47.507568 25050 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:47.568159 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:47.509259 25337 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:47.567500 23389 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:47.511945 24011 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:47.517798 25232 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:47.587296 23219 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:47.596501 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:47.494999 25076 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.592090 24899 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:47.592474 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.596350 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:47.601009 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.602041 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.603394 23469 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:47.540920 25250 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.597554 23219 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:47.507433 25265 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.589485 25237 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:47.586411 25175 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:47.626533 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:47.586216 25148 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:13:47.604526 25090 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:47.604247 25089 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:47.636090 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 153039us
W20260709 12:13:47.661798 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.541s (SENT)
I20260709 12:13:47.662374 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.590337 25163 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:13:47.627755 25250 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:47.632998 25125 raft_consensus.cc:2764] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:13:47.667757 24899 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:47.630385 25266 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:47.667737 25175 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:47.730165 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:47.666472 25090 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:47.750002 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 88239.3us
I20260709 12:13:47.737875 25177 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:47.753844 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.551s (SENT)
I20260709 12:13:47.755221 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:47.744123 25175 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:47.769764 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 201316us
I20260709 12:13:47.776302 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.776643 24899 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:47.753194 25076 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.801702 25265 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.762398 25239 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.805769 25239 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:47.809163 25342 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:13:47.811223 25264 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.811868 25264 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.821350 25264 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:47.829984 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 79305.5us
I20260709 12:13:47.829668 25090 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:47.830835 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.940s (SENT) [suppressed 52 similar messages]
W20260709 12:13:47.836994 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 235037us
I20260709 12:13:47.837864 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.839414 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:47.838506 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 292763us
I20260709 12:13:47.850862 24011 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:47.856470 24013 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:13:47.856945 24013 pending_rounds.cc:85] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 4
I20260709 12:13:47.857250 24013 pending_rounds.cc:107] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:13:47.857578 24013 raft_consensus.cc:2889] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:47.829568 25232 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.860153 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:47.860913 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:47.821563 25344 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:13:47.858965 25264 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:47.859193 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.787891 25050 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.872779 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:47.855893 25076 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:47.894410 24899 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:47.863903 25261 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:47.861519 25336 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 12:13:47.923583 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.897693 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:47.914057 25355 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.897760 25264 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:47.921070 25076 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:13:47.938510 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 347775us
I20260709 12:13:47.906227 25177 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:47.919503 25266 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:47.920742 25163 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:13:47.885147 25050 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:47.937656 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 157412us
I20260709 12:13:47.931358 25261 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:47.941999 25053 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.943696 25350 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:47.944793 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:47.951655 25257 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:47.951732 25349 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:47.952836 25148 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:47.944736 25264 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.002882 25349 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.944792 25076 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.957804 24899 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.958487 25363 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:47.961767 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:47.964104 25089 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:47.931981 24008 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:47.966560 25175 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.967053 25125 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:13:47.971230 25114 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.710s	user 0.010s	sys 0.000s
I20260709 12:13:47.936118 25354 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:13:47.974992 23321 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:47.981637 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.983193 23317 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:47.987726 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.937300 23388 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:47.939630 25355 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.994145 25262 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:47.995419 25245 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:47.996979 25266 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.014171 25089 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.008945 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.008966 24899 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:47.989636 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:47.896184 25337 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:48.016525 25250 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.016909 25261 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.052233 23321 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:48.032162 25239 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:47.999467 25050 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.043172 25352 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:48.068521 25352 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:48.068953 25352 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.070240 25250 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.043335 25053 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.045586 25089 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.046145 25349 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.046767 25262 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.047319 25245 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:48.077848 25245 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:48.079164 25245 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.080855 25245 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.048817 25237 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.022960 25363 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.045130 25357 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:48.051369 25232 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.052277 25337 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.011267 25257 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.021023 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.066047 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:48.066978 24899 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.093393 23321 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:48.091114 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:48.094110 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 221437us
I20260709 12:13:48.094658 24899 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:48.094791 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.095019 24899 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:48.095355 24899 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.104209 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.083815 25357 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.125494 25358 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:48.085938 25114 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:13:48.130751 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 299935us
W20260709 12:13:48.141995 23434 leader_election.cc:336] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.661s (SENT)
I20260709 12:13:48.104884 25177 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:48.110474 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:48.156227 25363 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.150460 25351 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.158362 25351 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.160730 25351 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.162827 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.140585 25114 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.183633 24899 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:48.187915 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45974.4us
W20260709 12:13:48.191679 23434 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.687s (SENT)
I20260709 12:13:48.192312 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:13:48.193156 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 205400us
I20260709 12:13:48.174335 25360 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:48.168404 25177 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.205960 24899 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.191339 25351 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.207475 25264 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:13:48.209561 25350 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:48.210002 25350 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:48.211095 25209 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:48.210991 25350 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.211794 25209 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.212589 25350 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.213701 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:48.215143 25209 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:48.216782 23434 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.737s (SENT)
I20260709 12:13:48.217360 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:48.218340 25209 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:48.219842 25209 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.198599 25265 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:48.220461 25209 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.224660 25209 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.227461 25209 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:48.232041 25352 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:48.241600 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 296872us
I20260709 12:13:48.251210 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:48.202189 25360 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:48.254271 23462 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:48.262096 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:48.265697 23467 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:48.266697 24007 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:48.266808 23392 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:13:48.267381 25257 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.268618 23462 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:48.276121 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.284608 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.284869 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:48.286289 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.265698 25090 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.290642 23463 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.291217 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:48.257928 25360 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.293574 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.254448 25125 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:13:48.299443 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:48.300846 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:48.238257 25114 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.301653 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.301887 23394 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.302145 25114 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.302873 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.303136 24007 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.303586 23396 raft_consensus.cc:2368] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:13:48.254719 25250 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:13:48.304986 25250 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:48.305331 25250 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:48.306260 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.307376 24015 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:13:48.307567 25352 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.308334 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:48.308909 24015 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.269457 25237 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.309937 25265 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.310168 24016 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.296064 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.311221 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.312023 25089 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.312745 24014 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:13:48.291631 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 170226us
I20260709 12:13:48.312021 24016 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:13:48.293867 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 155827us
I20260709 12:13:48.314481 23392 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.306143 25250 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.295507 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:48.296876 25257 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:48.311331 24011 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:48.318035 23462 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:48.194031 25357 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.326246 23463 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:48.294966 25355 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.329627 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.335811 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.336721 24007 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:48.337827 24010 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:48.337977 25250 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.338860 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.339609 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.340376 23390 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:48.271128 25209 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:13:48.349565 25364 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:48.350199 25364 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.310025 25177 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.352919 25364 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.287431 25090 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:48.371160 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50797.6us
I20260709 12:13:48.372109 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:48.405840 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.399652 23434 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:48.380864 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.399850 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.417918 23393 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:13:48.418299 24899 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:48.363077 25265 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:13:48.418013 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45970.6us
I20260709 12:13:48.411643 23390 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:48.419376 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:48.432125 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:48.442765 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.444376 24015 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:48.430702 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:48.447257 24899 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:48.447732 24899 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.448051 24899 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.370481 25261 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:48.453742 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 149816us
I20260709 12:13:48.377843 25053 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.325253 25369 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:13:48.377044 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:48.470127 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.417773 25245 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.421108 25257 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:48.471849 23469 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:48.473712 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:48.526849 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.527004 23392 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:48.380079 25357 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.474334 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.474975 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.505682 23387 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 2: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:13:48.499552 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.529260 23390 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.506978 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.529852 23394 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:48.444092 25364 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:13:48.507314 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:48.537416 23317 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:48.450223 25237 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:13:48.535307 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 169226us
I20260709 12:13:48.544270 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.463622 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.543256 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.545483 23316 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:48.547487 23395 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.515269 24007 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:48.551020 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.514740 25265 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.515014 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:48.563242 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49355.6us
I20260709 12:13:48.563820 23317 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:48.565060 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:48.526257 25257 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:48.568210 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:48.473820 25053 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.568884 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.569923 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.569147 24010 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:48.566759 25265 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.575280 25265 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.575843 25265 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:48.576206 25265 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:48.577486 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.578433 23317 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:13:48.583297 24011 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:48.518563 25245 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.556905 23312 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:48.593959 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:48.597296 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.599045 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.592731 24899 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.568994 25209 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:48.612214 25245 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.614583 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.615075 25209 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.615586 25209 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.617600 25148 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:48.617998 25148 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.618352 25148 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.621289 25362 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:13:48.622177 25370 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:13:48.615805 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.582998 25337 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:48.576577 25265 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:48.592969 25351 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:48.589771 25336 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.626127 23317 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:48.636102 23317 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:48.623792 23387 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:48.628954 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.638110 23316 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:48.635398 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.606964 25261 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.638947 23469 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:48.641793 23463 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:48.648949 24899 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.608110 25175 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:48.636859 23317 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.651219 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.654351 23392 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
W20260709 12:13:48.663050 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118898us
I20260709 12:13:48.606608 25264 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:13:48.632946 25351 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.668542 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.672398 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.673062 23387 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.669190 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.669648 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:48.633334 25209 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.675609 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.632162 25337 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.650403 25383 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:48.682780 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.660377 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:48.685504 23313 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 1.
I20260709 12:13:48.683518 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.686744 23395 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.666745 25090 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.694103 25382 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:13:48.674654 25351 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.653561 25336 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:48.695204 25392 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:13:48.696367 25266 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:48.697597 23313 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:13:48.697885 25377 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:13:48.701421 23387 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:48.694429 25384 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.709911 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.777264 23392 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:48.777551 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 113689us
I20260709 12:13:48.778628 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.653035 25257 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.707012 23395 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:48.674248 25362 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:13:48.713718 24899 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:48.773679 25266 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.712564 25209 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.709640 25163 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:48.789448 25379 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:13:48.773710 24010 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.776242 23313 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:48.791587 25266 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:48.807663 25175 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:48.810034 23313 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.812003 24899 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:48.820114 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.798065 25257 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.830333 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:48.831060 24012 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.790591 25384 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.834479 24007 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:48.835969 23313 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:48.842057 23391 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
W20260709 12:13:48.845749 25394 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:13:48.860898 24010 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:48.862169 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.863889 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.863976 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:48.864476 25381 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
I20260709 12:13:48.864737 25232 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:48.846793 25391 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:13:48.850049 25389 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:13:48.795316 25209 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.851585 23392 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:48.860958 25351 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:48.867967 25209 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:48.868855 25396 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:13:48.797112 25163 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.865175 25232 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.885681 23393 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:48.870230 23469 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:48.891053 23390 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:48.831732 25395 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:48.893893 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.893697 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:48.894256 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.894747 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:48.896546 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.897337 23390 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:48.898655 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.899634 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:48.902738 25245 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:48.903414 25373 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:48.903930 25373 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.904779 25163 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.905798 25053 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.906061 25373 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:48.906392 23392 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:48.909137 24012 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:48.909778 25337 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.905963 25148 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.910970 23387 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:48.910832 25411 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:13:48.912652 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1309) took 1021 ms. Trace:
I20260709 12:13:48.913007 23389 rpcz_store.cc:276] 0709 12:13:47.891253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.905830 (+ 14577us) service_pool.cc:224] Handling call
0709 12:13:47.927846 (+ 22016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.108233 (+180387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.108267 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.108894 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.286040 (+177146us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:48.286075 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.286137 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.863787 (+577650us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:13:48.902020 (+ 38233us) raft_consensus.cc:1685] finished
0709 12:13:48.902041 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.912588 (+ 10547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2555,"raft.queue_time_us":30850,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":69,"spinlock_wait_cycles":37183744,"thread_start_us":350809,"threads_started":2,"wal-append.queue_time_us":344086}
I20260709 12:13:48.914307 25148 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:48.917246 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.887018 25232 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.884631 25351 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:48.917019 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.889901 25371 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:48.922685 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 144158us
I20260709 12:13:48.928647 23390 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:48.927598 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:48.951573 25413 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:48.951676 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.952514 24012 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:13:48.952900 25376 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:13:48.955423 25412 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:48.956252 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 550) took 1389 ms. Trace:
I20260709 12:13:48.900681 25395 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.932446 25175 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.955667 25383 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.930191 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:48.960471 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:48.964946 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:48.965813 23462 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:48.963869 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:48.972255 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:48.959610 25209 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:48.952445 25239 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.977195 23465 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:48.968878 25360 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.004s
I20260709 12:13:48.980717 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:48.963284 25383 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:48.983139 23468 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:48.975797 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:48.991765 24016 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:48.985369 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55283.2us
I20260709 12:13:48.962657 25053 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:48.979597 23462 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.962502 25337 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:48.968252 23461 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:13:48.995977 25239 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.998051 25148 raft_consensus.cc:476] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260709 12:13:48.998380 23317 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:48.998636 25395 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:48.999740 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 70144.8us
I20260709 12:13:48.928596 25350 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:13:48.913158 25265 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:48.949687 25373 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:13:48.985843 25400 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:49.027086 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63308.8us
I20260709 12:13:49.028291 23321 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:48.987330 25371 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:49.036968 23321 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.006834 25337 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:49.037956 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.043046 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.037119 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.047487 23465 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:49.027218 25420 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:49.034353 25125 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:13:49.044616 23321 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.057766 25419 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:49.059508 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.035560 25245 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:49.060526 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.061875 23321 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.042433 25371 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.043705 25351 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.007324 25360 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:49.068289 23320 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:49.063799 23317 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:13:49.020469 25262 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:13:49.048892 25337 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:49.066722 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.066955 25419 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:49.072072 25411 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:49.074575 23316 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:49.074887 23321 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:48.994552 25407 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 12:13:49.082849 23388 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:49.057195 25053 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:49.068194 25371 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.064664 25413 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:49.036707 25400 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:13:49.091624 25354 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:49.095124 25384 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.097010 25406 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:49.071298 25420 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.101703 25355 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:13:49.154215 23316 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:49.104272 25418 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.073342 25351 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.150027 23313 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:13:49.112131 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52698.2us
I20260709 12:13:49.170225 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.114420 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.097517 25415 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:49.118474 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 88847.7us
I20260709 12:13:49.120864 25053 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.122763 25266 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.128822 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:13:49.130084 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60495.1us
I20260709 12:13:49.134117 25360 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.137674 25404 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.116667 25354 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.141999 25406 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.143638 25245 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.144513 25239 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:48.956640 24009 rpcz_store.cc:276] 0709 12:13:47.567015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.592040 (+ 25025us) service_pool.cc:224] Handling call
0709 12:13:47.950360 (+358320us) spinlock_profiling.cc:243] Waited 991 us on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:47.950614 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.951051 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:47.951086 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.041047 (+ 89961us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:13:48.045964 (+  4917us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.444206 (+398242us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:48.586144 (+141938us) log.cc:844] Serialized 14 byte log entry
0709 12:13:48.610494 (+ 24350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.610588 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.952231 (+341643us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:13:48.956015 (+  3784us) raft_consensus.cc:1685] finished
0709 12:13:48.956037 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.956167 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3690,"prepare.queue_time_us":144413,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":2812,"raft.queue_time_us":14331,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":13363,"spinlock_wait_cycles":70477824,"thread_start_us":422361,"threads_started":2,"wal-append.queue_time_us":600716}
I20260709 12:13:49.198536 25404 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.161103 24016 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:49.200881 25404 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:49.107446 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.222555 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:49.282702 25368 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:49.194818 23395 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:49.186856 23287 leader_election.cc:400] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:13:49.345335 23287 leader_election.cc:403] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:49.346038 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 162133us
I20260709 12:13:49.315143 25030 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:49.300774 25232 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.193677 25266 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.313164 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.184557 25209 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:49.356128 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.182447 25418 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.241405 25422 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:13:49.249661 25406 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.300246 25336 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.248658 25383 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.248950 25354 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:49.241047 25386 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.677s	user 0.002s	sys 0.002s
I20260709 12:13:49.186002 25395 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:13:49.326751 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 104338us
I20260709 12:13:49.241132 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.249603 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.345111 25368 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.295468 25413 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:49.246102 25261 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.596s	user 0.007s	sys 0.004s
I20260709 12:13:49.430939 25261 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:49.223649 25371 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.433116 25371 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:49.281167 25374 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.288511 25337 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:49.333360 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91428.9us
I20260709 12:13:49.434106 23312 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:49.321476 23319 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:49.433876 25374 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.436895 23319 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.436786 23312 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.337852 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:49.338497 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 143310us
I20260709 12:13:49.229746 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.450907 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.451069 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:49.450999 23312 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.452127 23312 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:49.452461 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.453341 25423 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:49.453596 25261 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:49.454885 25261 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:49.168430 25420 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.458016 25420 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.229235 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.458889 23315 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:49.459725 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:49.336582 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:49.461517 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:49.461838 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 125380us
I20260709 12:13:49.355145 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.314059 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.356982 25030 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.360613 25125 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:13:49.464169 25395 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:49.465503 25395 raft_consensus.cc:1064] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:49.466261 25030 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:49.335412 25264 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:49.361266 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.368031 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.369407 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.190949 25415 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.468894 23320 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:49.471453 25415 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.319972 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2115) took 1061 ms. Trace:
I20260709 12:13:49.469678 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.396559 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 329) took 1165 ms. Trace:
I20260709 12:13:49.473536 23466 rpcz_store.cc:276] 0709 12:13:48.231552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.231735 (+   183us) service_pool.cc:224] Handling call
0709 12:13:48.232161 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.232619 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.232658 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.233254 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.358774 (+125520us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:48.358813 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.359462 (+   649us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:49.193307 (+833845us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:13:49.396215 (+202908us) raft_consensus.cc:1685] finished
0709 12:13:49.396239 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:49.396462 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90,"raft.queue_time_us":1417,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":11377,"spinlock_wait_cycles":45474048,"thread_start_us":125094,"threads_started":1,"wal-append.queue_time_us":237226}
I20260709 12:13:49.398859 25232 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:49.394397 25368 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.476214 25368 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.394289 25383 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.388605 25266 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:13:49.365317 23464 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:49.395439 25354 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:49.367040 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.403342 25418 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.463789 25371 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:49.357128 25432 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.389086 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.501624 23390 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:13:49.404295 25265 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.389922 25209 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.383661 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.363456 25425 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:13:49.437904 23319 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.268437 25351 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.438860 25374 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.439121 23467 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:13:49.430979 24007 raft_consensus.cc:1217] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5]   Dedup: 4.5->[]
I20260709 12:13:49.440580 25337 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.441201 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:49.463214 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108136us
W20260709 12:13:49.521828 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.357623 25373 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:49.471608 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:49.523943 23320 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:13:49.524132 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.472110 23388 rpcz_store.cc:276] 0709 12:13:48.258146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.258289 (+   143us) service_pool.cc:224] Handling call
0709 12:13:48.258670 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.087222 (+828552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:49.319700 (+232478us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54002c7254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:49.319899 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1895044352}
I20260709 12:13:49.524802 23390 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:13:49.470424 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.486368 23387 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:49.486925 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:49.489639 23315 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:49.492769 23394 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:13:49.526714 23391 raft_consensus.cc:2368] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:49.494578 23317 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:49.497301 23393 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.385010 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.499040 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.531241 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.532122 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.532806 23468 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:49.523943 25374 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:49.535501 25374 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:49.537793 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.537921 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:49.492947 25250 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:13:49.499029 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.500360 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.483345 25423 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:49.483180 25415 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.543259 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:49.543201 23320 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:49.501863 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.502877 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:49.508214 24899 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.544603 23389 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:49.544715 24899 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.540756 25423 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.545938 25423 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:49.546732 24899 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:49.488276 25355 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:49.490389 25420 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:49.509157 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.550396 25374 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.518921 23319 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:49.492347 25264 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.463831 25350 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:49.539680 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:49.557371 25350 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:13:49.557665 25441 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:13:49.504864 25125 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:13:49.540645 25030 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:13:49.559135 25408 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:49.567045 23467 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:49.499015 25419 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:13:49.548283 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:49.548867 25355 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.554103 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:49.557789 25350 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.557222 25089 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:49.578001 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.554171 25425 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:49.584719 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:49.586287 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.586936 23467 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:49.507861 25432 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.586880 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:49.585579 23463 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:49.587992 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.590375 23396 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:49.590868 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.594830 24016 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:49.467939 25395 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.604352 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:49.605360 24899 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.606106 24899 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.608156 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:49.551999 25415 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.586439 25337 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:49.543532 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 54318.1us
I20260709 12:13:49.601234 25030 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:49.614385 23467 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:49.615681 23465 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:13:49.589020 25418 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:49.616484 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.611634 25089 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.620229 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:49.620052 25089 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.620908 23393 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:49.621802 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.622107 25089 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.622411 23391 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:49.600951 25413 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:49.589067 25239 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:13:49.635617 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:49.635510 25264 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.652518 23396 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.610241 25363 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.630798 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.614269 25351 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.618583 25418 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.646802 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
W20260709 12:13:49.638018 25444 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:13:49.676960 23466 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:13:49.646572 25415 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:49.679651 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 75373.2us
W20260709 12:13:49.672770 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53701.2us
W20260709 12:13:49.667404 25438 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:13:49.645535 25337 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:49.639144 25355 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.614533 25425 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:49.703758 25425 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.681041 25089 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:13:49.709667 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.710510 23316 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:49.710902 23316 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.711606 23316 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.625049 25400 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 12:13:49.684809 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.681383 25404 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:13:49.685685 25407 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:13:49.686316 25336 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:49.716954 25336 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.717352 25336 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:49.638247 23312 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:13:49.617676 24009 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:49.646107 25350 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.698596 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:49.712491 23316 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:49.639958 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:49.710552 23468 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:49.677011 25363 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.630381 25349 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:13:49.680126 25418 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:49.723575 23312 pending_rounds.cc:85] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 2
I20260709 12:13:49.624940 25250 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:13:49.755641 23396 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:49.756173 23466 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.728603 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:49.722940 23319 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:13:49.758522 23319 pending_rounds.cc:85] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 1
I20260709 12:13:49.758823 23319 pending_rounds.cc:107] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:13:49.731527 23461 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:49.683635 25351 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.779274 23463 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:49.712486 23314 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:49.725752 25358 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:49.759254 23319 raft_consensus.cc:2889] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.759389 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:49.760252 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:49.762708 24008 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:49.765630 25445 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:49.765865 25125 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:49.765187 23316 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:13:49.769501 25350 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:49.769905 25030 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.772644 25363 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.746429 25337 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.775296 23466 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:49.757542 23312 pending_rounds.cc:107] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:13:49.790803 23316 pending_rounds.cc:85] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:13:49.793099 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:49.795823 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.786628 25358 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.799672 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.800148 25373 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:13:49.802551 25351 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:49.732436 25400 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:49.824721 24010 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:49.748257 25355 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:13:49.821220 25349 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:49.826335 25349 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.827291 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.831624 24013 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:49.833026 24013 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:49.805250 23312 raft_consensus.cc:2889] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.804742 23316 pending_rounds.cc:107] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:13:49.805544 23394 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:13:49.806896 23396 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:13:49.808159 23393 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.837818 25355 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.819850 25363 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:49.838296 25355 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.838793 25363 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.840934 23316 raft_consensus.cc:2889] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:49.839713 25363 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:49.809358 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.810335 23389 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:49.810865 23391 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:49.811911 23395 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:49.814918 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.815428 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.846761 24011 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:49.847565 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.848335 25412 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:49.848913 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.816254 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:49.805122 25352 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:13:49.851212 23463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:39816
I20260709 12:13:49.851593 25257 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:49.851806 23463 raft_consensus.cc:606] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:49.852316 23463 raft_consensus.cc:614] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:49.818871 24016 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:13:49.792814 25250 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:49.861610 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.782874 23314 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.823042 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.823276 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.805087 25354 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:49.845813 25349 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.816619 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:49.874130 25354 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:49.874645 25354 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:49.878348 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:49.878979 23320 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.878611 23314 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.879336 23320 raft_consensus.cc:614] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:49.879679 23314 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:49.860251 25239 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:13:49.883935 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.884543 23315 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:49.884948 23315 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:49.857039 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.809123 25445 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.858752 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:49.883786 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:13:49.893477 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 70280.5us
I20260709 12:13:49.863916 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.894271 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:49.895691 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.861402 23218 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:49.898139 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 4, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 4 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:49.869329 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:49.892153 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.891578 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:49.894275 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:49.918247 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.809607 25030 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.925537 23315 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:49.858778 25355 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:49.819124 25367 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:13:49.940490 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:49.985635 23315 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:49.941027 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.989476 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.986063 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60068.1us
W20260709 12:13:49.993940 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53563.1us
I20260709 12:13:49.930516 25089 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:49.923295 25413 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:13:49.996837 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.994629 23315 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:13:49.998495 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.998804 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:49.934445 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:49.841675 25336 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:49.928650 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:13:50.000762 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66446.8us
I20260709 12:13:50.005190 23287 leader_election.cc:400] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:13:50.005599 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:50.006270 23315 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:50.002485 25373 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:13:49.908272 25239 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:49.923384 25351 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:49.928572 25053 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:50.020267 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:50.020843 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:50.022509 24015 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:13:49.996279 25232 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:49.995615 25413 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.001252 25262 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:50.028826 23318 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:50.000698 25403 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:49.990584 25404 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:13:50.032614 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:49.991019 25407 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:13:50.005522 25425 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.048652 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:50.049454 23319 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:13:50.051024 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 122469us
I20260709 12:13:50.037586 23315 raft_consensus.cc:606] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:50.020000 25089 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:49.991381 25445 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.060408 23219 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 314) took 3121 ms. Trace:
I20260709 12:13:50.058597 23319 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:50.061115 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:50.061358 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:50.020843 25351 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.002866 25423 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.028223 25232 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.033967 25413 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:50.011425 25428 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:13:50.080085 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:50.059473 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57195.9us
W20260709 12:13:50.090052 25449 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:13:50.091063 25408 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:50.091523 25408 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.091941 25408 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.092229 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:50.093593 25446 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.081881 25148 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.094168 25446 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.082098 25232 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.081666 25351 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:50.062340 25175 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:50.071266 23219 rpcz_store.cc:276] 0709 12:13:46.930794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.930992 (+   198us) service_pool.cc:224] Handling call
0709 12:13:46.983499 (+ 52507us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a65500b2 00007ff0af4c9759 00007ff0af461a76 00007ff0af432d58 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc
0709 12:13:50.052179 (+3068680us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:48.007095 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:48.007839 (+   744us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:48.007887 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:13:48.007918 (+    31us) tablet.cc:662] Decoding operations
0709 12:13:48.032179 (+ 24261us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:48.032241 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:48.032268 (+    27us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:13:48.033109 (+   841us) tablet.cc:701] Row locks acquired
0709 12:13:48.033123 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:13:48.033274 (+   151us) write_op.cc:270] Start()
0709 12:13:48.033390 (+   116us) write_op.cc:276] Timestamp: P: 1783599228033221 usec, L: 0
0709 12:13:48.033410 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:13:48.034264 (+   854us) log.cc:844] Serialized 4217 byte log entry
0709 12:13:48.154278 (+120014us) op_driver.cc:464] REPLICATION: finished
0709 12:13:48.562659 (+408381us) write_op.cc:301] APPLY: starting
0709 12:13:48.567710 (+  5051us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:48.607552 (+ 39842us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:48.659990 (+ 52438us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:48.830034 (+170044us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:48.830266 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:13:48.830309 (+    43us) write_op.cc:312] APPLY: finished
0709 12:13:48.830602 (+   293us) log.cc:844] Serialized 77 byte log entry
0709 12:13:48.830763 (+   161us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:48.830905 (+   142us) write_op.cc:454] Released schema lock
0709 12:13:48.830948 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:48.831331 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:48.831355 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":614,"reported_tablets":18,"spinlock_wait_cycles":52053248,"child_traces":[["op",{"apply.queue_time_us":332662,"apply.run_cpu_time_us":4548,"apply.run_wall_time_us":295342,"num_ops":10,"prepare.queue_time_us":277305,"prepare.run_cpu_time_us":4857,"prepare.run_wall_time_us":27464,"replication_time_us":99047,"thread_start_us":292151,"threads_started":2}]]}
I20260709 12:13:50.036391 25390 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:13:50.153836 23469 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:50.034861 25053 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.144101 25175 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.184485 25446 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:50.186342 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.047655 25403 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.180737 25175 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:50.181237 25404 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:50.217747 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 205467us
I20260709 12:13:50.274473 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:50.216687 25089 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:50.266217 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.257831 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:50.277817 23315 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:50.278219 23315 raft_consensus.cc:614] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:50.257691 25458 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:13:50.219723 25257 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:50.207535 25239 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.216369 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:50.255818 25447 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.649s	user 0.004s	sys 0.000s
I20260709 12:13:50.256443 25030 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:50.302971 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:50.308848 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:13:50.289950 25390 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:50.310145 25445 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:50.378213 25423 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:50.617730 24899 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:50.378914 25232 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:50.333750 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.633527 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.633991 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.224836 25404 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.260002 25400 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:13:50.395361 25053 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:50.527477 23314 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.000s
I20260709 12:13:50.277921 25336 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:50.524156 25350 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.721s	user 0.005s	sys 0.003s
W20260709 12:13:50.301019 25450 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
W20260709 12:13:50.396184 25456 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:13:50.380555 25351 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.521764 25403 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:50.395884 25418 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
W20260709 12:13:50.563038 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 288666us
I20260709 12:13:50.563791 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:50.571218 25264 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:50.576123 25427 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:50.578927 25432 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:50.579628 25250 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
W20260709 12:13:50.595288 25408 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.503s	user 0.007s	sys 0.001s
I20260709 12:13:50.597702 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:50.597985 25413 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 12:13:50.602401 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:50.589143 23469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 344) took 1279 ms (client timeout 1663 ms). Trace:
W20260709 12:13:50.593158 25412 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.000s
W20260709 12:13:50.510206 25354 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.001s
W20260709 12:13:50.715590 23981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.006s	sys 0.024s
I20260709 12:13:50.614820 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:50.306958 25239 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.625730 24899 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.626931 23467 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:50.651667 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 350) took 1067 ms. Trace:
W20260709 12:13:50.634941 25398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.017s	sys 0.033s
I20260709 12:13:50.693845 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:50.697528 23317 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:13:50.679504 25367 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:50.715581 23469 rpcz_store.cc:269] 0709 12:13:49.310029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.313777 (+  3748us) service_pool.cc:224] Handling call
0709 12:13:50.588873 (+1275096us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54003dac54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:50.589038 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587919104}
I20260709 12:13:50.711248 23314 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:13:50.720602 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2132) took 1181 ms. Trace:
W20260709 12:13:50.561033 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.723340 23981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.007s	sys 0.024s
I20260709 12:13:50.696424 25400 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:50.648031 25390 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.696046 25427 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:50.698101 25445 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:50.718490 25262 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.018s	sys 0.042s
W20260709 12:13:50.715303 24008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.021s	sys 0.066s
W20260709 12:13:50.731992 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.874027 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1562) took 1291 ms. Trace:
W20260709 12:13:50.732715 25428 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.021s	sys 0.040s
I20260709 12:13:50.732586 23470 rpcz_store.cc:276] 0709 12:13:49.583912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.584149 (+   237us) service_pool.cc:224] Handling call
0709 12:13:49.593534 (+  9385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.617420 (+ 23886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.617468 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.623375 (+  5907us) log.cc:844] Serialized 26 byte log entry
0709 12:13:49.760754 (+137379us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:49.760792 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.626247 (+865455us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b54000c8854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:50.626313 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.626372 (+    59us) raft_consensus.cc:1685] finished
0709 12:13:50.626403 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.651559 (+ 25156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27702,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":128,"spinlock_wait_cycles":49087360,"thread_start_us":137053,"threads_started":1,"wal-append.queue_time_us":194255}
W20260709 12:13:50.739962 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46234.1us
W20260709 12:13:50.885390 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.736423 23319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.000s	sys 0.056s
W20260709 12:13:50.886015 23319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.000s	sys 0.057s
I20260709 12:13:50.719082 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:50.736802 23317 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:13:50.887950 23317 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:13:50.888296 23317 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:50.721208 25350 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:50.731904 24899 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.855574 23392 rpcz_store.cc:276] 0709 12:13:49.538775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.538899 (+   124us) service_pool.cc:224] Handling call
0709 12:13:49.550637 (+ 11738us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.550992 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.551021 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.551497 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:13:49.606415 (+ 54918us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:49.606458 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:49.606524 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.255293 (+648769us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:13:50.714149 (+458856us) raft_consensus.cc:1685] finished
0709 12:13:50.714181 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.720516 (+  6335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"raft.queue_time_us":631,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":9582720,"thread_start_us":54547,"threads_started":1,"wal-append.queue_time_us":54885}
W20260709 12:13:50.700796 25373 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.060s	sys 0.129s
I20260709 12:13:50.722685 24012 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:50.728343 25412 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:50.739238 25398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.017s	sys 0.033s
I20260709 12:13:50.854138 23314 pending_rounds.cc:85] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:13:50.702396 25264 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.728883 25354 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.702808 25418 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:50.879650 24008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.021s	sys 0.066s
I20260709 12:13:50.874491 23468 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:50.713323 25250 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:50.740602 25432 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:50.945344 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:50.896390 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:50.720305 25413 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.891350 25415 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:50.864336 25403 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.910389 25257 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:50.880146 23462 rpcz_store.cc:276] 0709 12:13:49.582503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.582637 (+   134us) service_pool.cc:224] Handling call
0709 12:13:49.582927 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.589516 (+  6589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.589546 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.601358 (+ 11812us) log.cc:844] Serialized 26 byte log entry
0709 12:13:49.783491 (+182133us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:49.783531 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:49.783615 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.873462 (+1089847us) raft_consensus.cc:1685] finished
0709 12:13:50.873621 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.873946 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1374,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":159482,"threads_started":1,"wal-append.queue_time_us":209155}
W20260709 12:13:50.872779 23318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.019s	sys 0.037s
W20260709 12:13:50.956342 23318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.019s	sys 0.038s
W20260709 12:13:50.877674 25262 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.018s	sys 0.043s
I20260709 12:13:50.873126 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 566) took 1329 ms. Trace:
I20260709 12:13:50.873847 25423 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:50.884169 25428 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.021s	sys 0.040s
I20260709 12:13:50.931900 24899 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:50.911872 25373 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.062s	sys 0.133s
W20260709 12:13:50.965420 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 69146.6us
I20260709 12:13:50.911237 23314 pending_rounds.cc:107] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 12:13:50.942481 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.021s	sys 0.075s
I20260709 12:13:50.729584 25175 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.891067 25350 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:50.892108 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 443) took 1025 ms. Trace:
I20260709 12:13:50.865674 25367 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.716410 25408 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.953058 24014 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:13:50.880374 25445 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:50.872937 25472 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
W20260709 12:13:50.889746 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.038s	sys 0.060s
I20260709 12:13:50.931389 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 565) took 1391 ms. Trace:
W20260709 12:13:50.969058 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.021s	sys 0.075s
I20260709 12:13:50.970360 23462 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:13:50.916697 25400 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:13:50.974166 23463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.805s	user 0.011s	sys 0.058s
I20260709 12:13:50.967784 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:50.960996 25363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.005s	sys 0.046s
I20260709 12:13:50.854425 25358 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:50.972298 25337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.008s	sys 0.017s
I20260709 12:13:50.994254 24899 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:50.985090 23464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.004s	sys 0.015s
I20260709 12:13:50.971817 23314 raft_consensus.cc:2889] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:50.960785 25262 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:50.958761 25403 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.956640 25415 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.965238 25257 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:50.962208 25467 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
W20260709 12:13:51.158154 23463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.016s	sys 0.064s
W20260709 12:13:51.080616 23465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.816s	user 0.038s	sys 0.067s
I20260709 12:13:51.073760 23396 raft_consensus.cc:2410] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:13:51.085374 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 378319us
W20260709 12:13:51.157742 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.128s	user 0.039s	sys 0.065s
W20260709 12:13:51.079257 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.045s	sys 0.098s
W20260709 12:13:51.160573 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.046s	sys 0.098s
W20260709 12:13:51.159112 23465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.045s	sys 0.074s
W20260709 12:13:51.160269 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.162884 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:50.972720 25398 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:50.942795 25427 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.205634 24899 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:50.992986 25461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.006s	sys 0.026s
I20260709 12:13:51.214522 25408 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:50.975167 25250 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:51.166395 25363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.011s	sys 0.046s
I20260709 12:13:51.066800 25373 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:51.218845 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 56070.1us
I20260709 12:13:51.022413 25473 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.135064 25407 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:50.960939 24007 rpcz_store.cc:276] 0709 12:13:49.539749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.541043 (+  1294us) service_pool.cc:224] Handling call
0709 12:13:50.597560 (+1056517us) spinlock_profiling.cc:243] Waited 7.82 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:50.869096 (+271536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17975680}
I20260709 12:13:51.095501 25469 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.135725 25354 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.054075 25232 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:51.169297 23464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.008s	sys 0.021s
I20260709 12:13:51.019896 25445 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:50.968665 25423 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.394728 24899 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:51.398586 25374 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 12:13:51.219568 25461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.006s	sys 0.034s
I20260709 12:13:51.086148 25404 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.210489 25432 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.212101 25411 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.220268 25358 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:51.174706 25351 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.557334 25351 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.557948 25351 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:51.558669 25463 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:13:51.559938 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1569) took 1702 ms. Trace:
I20260709 12:13:51.560101 25351 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.560591 25351 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.392205 25114 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.560316 23466 rpcz_store.cc:276] 0709 12:13:49.857308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.877048 (+ 19740us) service_pool.cc:224] Handling call
0709 12:13:49.877388 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.924894 (+ 47506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.924925 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.927830 (+  2905us) log.cc:844] Serialized 26 byte log entry
0709 12:13:50.167341 (+239511us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:50.167398 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.167635 (+   237us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.559735 (+1392100us) raft_consensus.cc:1685] finished
0709 12:13:51.559756 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.559881 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17116,"raft.queue_time_us":17321,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":19304,"thread_start_us":154495,"threads_started":1,"wal-append.queue_time_us":717593}
I20260709 12:13:51.561852 25114 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:51.562889 25470 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:13:50.976323 23312 rpcz_store.cc:276] 0709 12:13:49.855240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.862353 (+  7113us) service_pool.cc:224] Handling call
0709 12:13:50.614685 (+752332us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:50.878206 (+263521us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16904576}
I20260709 12:13:51.564061 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 626) took 1750 ms. Trace:
W20260709 12:13:51.528450 23319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.003s	sys 0.036s
W20260709 12:13:51.564870 23319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.004s	sys 0.036s
I20260709 12:13:51.517478 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 439) took 1726 ms. Trace:
I20260709 12:13:51.565924 23314 rpcz_store.cc:276] 0709 12:13:49.709192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.712055 (+  2863us) service_pool.cc:224] Handling call
0709 12:13:49.712353 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.217658 (+1505305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.435418 (+217760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.432735 25239 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:51.396867 25481 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:51.567436 25481 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:51.568387 25481 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.564466 24008 rpcz_store.cc:276] 0709 12:13:49.813682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.817568 (+  3886us) service_pool.cc:224] Handling call
0709 12:13:49.850216 (+ 32648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.886805 (+ 36589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.894384 (+  7579us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.011714 (+117330us) log.cc:844] Serialized 26 byte log entry
0709 12:13:50.949366 (+937652us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:50.952172 (+  2806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.966521 (+ 14349us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.563843 (+597322us) raft_consensus.cc:1685] finished
0709 12:13:51.563868 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.563998 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9523,"raft.queue_time_us":12423,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":37092,"thread_start_us":856279,"threads_started":1,"wal-append.queue_time_us":1111730}
I20260709 12:13:51.564014 25114 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.561175 25351 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.573699 25351 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.587544 25411 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.142160 25350 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.590291 25350 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:51.524609 23461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.013s	sys 0.025s
I20260709 12:13:51.441759 25427 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:51.558374 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.029s	sys 0.053s
W20260709 12:13:51.603948 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.501s	user 0.029s	sys 0.054s
I20260709 12:13:51.524291 25445 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.602378 23318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.080s	sys 0.106s
W20260709 12:13:51.667049 23318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.081s	sys 0.106s
I20260709 12:13:51.392227 25250 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.205390 24012 rpcz_store.cc:276] 0709 12:13:49.526113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.527864 (+  1751us) service_pool.cc:224] Handling call
0709 12:13:50.602227 (+1074363us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:50.917119 (+314892us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19001600}
I20260709 12:13:51.441020 25390 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.432464 25373 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.667690 23318 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:13:51.551916 25264 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.431907 25398 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.684058 23462 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:51.686098 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:51.439666 25467 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:51.438232 25473 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.712248 23462 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:13:51.694644 23284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.015s	sys 0.017s
W20260709 12:13:51.714466 23284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.015s	sys 0.018s
W20260709 12:13:51.714994 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.715420 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:51.715843 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.666554 25474 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.811069 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:51.817394 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1575) took 1727 ms. Trace:
I20260709 12:13:51.817863 23469 rpcz_store.cc:276] 0709 12:13:50.089868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.883272 (+793404us) service_pool.cc:224] Handling call
0709 12:13:50.883616 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.898099 (+ 14483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.898160 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.898920 (+   760us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.434078 (+535158us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:51.434121 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.434189 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.817170 (+382981us) raft_consensus.cc:1685] finished
0709 12:13:51.817189 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.817330 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8654,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":12843,"thread_start_us":483139,"threads_started":1,"wal-append.queue_time_us":885100}
I20260709 12:13:51.819403 25477 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.820040 25477 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.821658 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1572) took 1956 ms. Trace:
I20260709 12:13:51.822053 23465 rpcz_store.cc:276] 0709 12:13:49.865187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.263389 (+398202us) service_pool.cc:224] Handling call
0709 12:13:50.263785 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.264158 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.264185 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.264675 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.161495 (+896820us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:51.161549 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.161616 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.819436 (+657820us) raft_consensus.cc:1685] finished
0709 12:13:51.819459 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.821596 (+  2137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1273,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":4963,"thread_start_us":896444,"threads_started":1,"wal-append.queue_time_us":1551224}
I20260709 12:13:51.823192 25266 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.823943 25266 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.824446 25464 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:51.825207 25464 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:51.826731 25266 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.828264 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:51.827831 25464 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.833118 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:51.833526 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:51.834410 23391 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:51.834911 23392 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:51.835198 23394 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:51.835261 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:51.679046 23461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.014s	sys 0.026s
I20260709 12:13:51.681125 25413 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:51.702298 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1571) took 1843 ms. Trace:
I20260709 12:13:51.880950 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:51.881963 23465 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7.
W20260709 12:13:51.850800 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.008s	sys 0.015s
W20260709 12:13:51.884464 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.008s	sys 0.015s
W20260709 12:13:51.885177 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 917546us
I20260709 12:13:51.803529 25175 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:51.851154 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.007s	sys 0.016s
W20260709 12:13:51.801038 25355 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.015s	sys 0.062s
I20260709 12:13:51.596809 25432 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:51.694684 25462 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.018s	sys 0.014s
I20260709 12:13:51.687826 25351 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.688274 25400 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.844877 23466 raft_consensus.cc:2368] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:51.702890 25265 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:51.671675 25469 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.444595 25407 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:51.903065 23470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.017s	sys 0.007s
W20260709 12:13:51.952411 23470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.018s	sys 0.007s
W20260709 12:13:51.903748 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.007s	sys 0.016s
I20260709 12:13:51.680023 25445 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.853462 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:51.889700 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:51.907436 23463 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.021s	sys 0.056s
W20260709 12:13:51.959295 23463 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.021s	sys 0.056s
I20260709 12:13:51.890779 25413 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:51.960166 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1570) took 2101 ms. Trace:
I20260709 12:13:51.707993 25467 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.665797 25262 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:51.946522 25462 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.019s	sys 0.014s
W20260709 12:13:51.939946 23316 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.032s	sys 0.016s
W20260709 12:13:51.967471 23316 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.032s	sys 0.016s
W20260709 12:13:51.945374 23287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.013s
W20260709 12:13:51.968148 23287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.002s	sys 0.013s
I20260709 12:13:51.968186 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:51.904587 25355 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.288s	user 0.015s	sys 0.063s
I20260709 12:13:51.950441 24899 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:13:51.946331 23286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.021s
W20260709 12:13:51.903393 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.001s	sys 0.023s
W20260709 12:13:51.971611 23286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.005s	sys 0.022s
W20260709 12:13:51.971750 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.001s	sys 0.023s
I20260709 12:13:51.954689 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1574) took 2028 ms. Trace:
I20260709 12:13:51.842576 25474 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.957613 25468 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.957791 25403 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.959024 23469 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:51.965797 23285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.019s	sys 0.047s
I20260709 12:13:51.906279 25367 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.810076 25398 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:51.924551 23433 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.015s	sys 0.009s
W20260709 12:13:51.968864 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 575901us
W20260709 12:13:51.991060 23433 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.015s	sys 0.009s
I20260709 12:13:51.946610 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:51.989812 25337 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:51.992846 25476 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:13:51.945029 25261 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:51.960667 23463 rpcz_store.cc:276] 0709 12:13:49.858358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.905776 (+ 47418us) service_pool.cc:224] Handling call
0709 12:13:49.906154 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.906369 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:50.143342 (+236973us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.959692 (+1816350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.959767 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:51.959833 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.959921 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.960095 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53398,"prepare.queue_time_us":129718,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":362157,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":1896038,"threads_started":3,"wal-append.queue_time_us":1532525}
I20260709 12:13:51.959085 25411 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:51.958801 25257 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:51.984401 23285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.019s	sys 0.047s
I20260709 12:13:51.963521 25467 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:51.992482 25337 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:51.995745 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.970525 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:51.993614 25499 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:51.993782 25261 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.001161 25495 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:52.001590 25489 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:52.001209 25462 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:52.057969 25482 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.980s	user 0.003s	sys 0.000s
I20260709 12:13:52.058130 25462 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.002480 25509 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:13:52.059939 25477 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.003181 25367 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.977306 23461 rpcz_store.cc:276] 0709 12:13:49.926072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.304115 (+378043us) service_pool.cc:224] Handling call
0709 12:13:50.723183 (+419068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.852561 (+129378us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:13:50.947730 (+ 95169us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.902123 (+954393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.917748 (+ 15625us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:51.944594 (+ 26846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.949693 (+  5099us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.954602 (+  4909us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7651,"prepare.queue_time_us":52238,"prepare.run_wall_time_us":38,"raft.queue_time_us":120824,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"spinlock_wait_cycles":21771648,"thread_start_us":885044,"threads_started":1,"wal-append.queue_time_us":996977}
I20260709 12:13:52.004562 25494 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:52.005736 25404 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.953564 25400 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.972167 25498 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:51.965641 25415 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:51.989591 25483 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:13:51.880443 23464 rpcz_store.cc:276] 0709 12:13:49.858592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.045874 (+187282us) service_pool.cc:224] Handling call
0709 12:13:50.102213 (+ 56339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.178138 (+ 75925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.180034 (+  1896us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.274478 (+ 94444us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.551605 (+1277127us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:51.599255 (+ 47650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.666639 (+ 67384us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.673651 (+  7012us) raft_consensus.cc:1685] finished
0709 12:13:51.678711 (+  5060us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.693921 (+ 15210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2868,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":6528,"thread_start_us":1244354,"threads_started":1,"wal-append.queue_time_us":938016}
I20260709 12:13:52.020370 24009 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:51.953333 25351 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:51.959028 25265 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:51.970001 25413 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.050558 23469 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:13:52.029096 25504 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.023149 25398 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.024353 25257 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:51.980758 25492 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 12:13:52.033208 25411 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:52.158860 25491 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:13:52.159185 25373 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.159359 25496 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.063438 25423 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.161999 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 461) took 2367 ms. Trace:
W20260709 12:13:52.164222 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 193798us
I20260709 12:13:52.164878 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2188) took 1472 ms. Trace:
I20260709 12:13:52.158941 25407 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.288684 23470 rpcz_store.cc:276] 0709 12:13:50.693037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.954561 (+261524us) service_pool.cc:224] Handling call
0709 12:13:50.973620 (+ 19059us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.973999 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.974042 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.974588 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.217098 (+242510us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b180036ef80. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:13:51.953033 (+735935us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:51.953094 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.953182 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.006937 (+ 53755us) raft_consensus.cc:1685] finished
0709 12:13:52.006957 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.164819 (+157862us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":30956,"spinlock_wait_cycles":71271680,"thread_start_us":735681,"threads_started":1,"wal-append.queue_time_us":983429}
W20260709 12:13:52.061939 23313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.015s	sys 0.037s
I20260709 12:13:51.990059 25468 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.160722 25469 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.164289 25250 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.174259 25432 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.175302 25498 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:52.398279 25498 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.060745 25499 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.402287 25474 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.300048 25266 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.403752 25266 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.410303 25473 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.412352 25261 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.414132 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1612) took 2523 ms. Trace:
I20260709 12:13:52.230285 25504 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.125741 25404 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.230064 25175 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:52.319309 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.319710 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:52.419062 23388 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.072439 25367 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:52.396520 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 108038us
I20260709 12:13:52.465191 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:52.287351 25398 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:52.395442 25501 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.268s	user 0.000s	sys 0.000s
I20260709 12:13:52.256069 25358 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.214s
I20260709 12:13:52.419169 23390 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:52.296790 25337 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.290779 25502 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:52.558730 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68718.9us
I20260709 12:13:52.414647 23317 rpcz_store.cc:276] 0709 12:13:49.890363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.891371 (+  1008us) service_pool.cc:224] Handling call
0709 12:13:49.891627 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.888852 (+997225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.413725 (+1524873us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b5400388454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:52.414009 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820341120}
I20260709 12:13:52.290398 24009 rpcz_store.cc:276] 0709 12:13:49.794920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.803075 (+  8155us) service_pool.cc:224] Handling call
0709 12:13:49.841551 (+ 38476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.989821 (+2148270us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b540049f0a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:13:52.161657 (+171836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.161925 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61107584}
I20260709 12:13:52.380287 25355 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.278302 25257 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:52.562358 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.027s	sys 0.079s
W20260709 12:13:52.599375 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.027s	sys 0.079s
I20260709 12:13:52.470861 25354 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:52.575809 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:52.578105 23318 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.884s	user 0.008s	sys 0.002s
I20260709 12:13:52.291379 25496 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.594741 25352 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:52.637413 25352 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:52.595512 25125 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:52.490139 25498 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:52.600157 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 654931us
I20260709 12:13:52.641742 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 444) took 2754 ms. Trace:
I20260709 12:13:52.642017 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:52.643575 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:52.568746 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 404005us
I20260709 12:13:52.490917 25432 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:52.606454 23359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.020s	sys 0.061s
W20260709 12:13:52.647023 23359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.021s	sys 0.061s
I20260709 12:13:52.610918 25468 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.613405 25367 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.615036 25266 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.620126 23388 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:13:52.624783 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2142) took 1918 ms (client timeout 2088 ms). Trace:
I20260709 12:13:52.630340 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2182) took 2854 ms. Trace:
I20260709 12:13:52.631453 25504 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.633522 25499 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.645920 24899 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:52.662174 24899 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:52.662544 24899 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.648466 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1264) took 2710 ms. Trace:
I20260709 12:13:52.567483 25462 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.472575 25175 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.582633 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2184) took 2792 ms. Trace:
I20260709 12:13:52.655656 23468 rpcz_store.cc:276] 0709 12:13:49.776190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.778873 (+  2683us) service_pool.cc:224] Handling call
0709 12:13:49.850351 (+ 71478us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:50.874128 (+1023777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.931161 (+ 57033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.630051 (+1698890us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400405bd0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:52.630254 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379827840}
W20260709 12:13:52.658610 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 494654us
W20260709 12:13:52.654871 23396 rpcz_store.cc:269] 0709 12:13:50.706542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.706706 (+   164us) service_pool.cc:224] Handling call
0709 12:13:52.624447 (+1917741us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54000a8554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:52.624690 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427383680}
I20260709 12:13:52.531267 25398 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.642315 23316 rpcz_store.cc:276] 0709 12:13:49.887812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.889171 (+  1359us) service_pool.cc:224] Handling call
0709 12:13:49.889476 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.891818 (+  2342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.891845 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.892293 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:13:50.315159 (+422866us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180035b720. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9
0709 12:13:51.437290 (+1122131us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b180035b720. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:13:51.967853 (+530563us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:51.967895 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.967959 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.641541 (+673582us) raft_consensus.cc:1685] finished
0709 12:13:52.641560 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.641686 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13735,"spinlock_wait_cycles":41578496,"thread_start_us":530368,"threads_started":1,"wal-append.queue_time_us":2115683}
I20260709 12:13:52.663599 23318 rpcz_store.cc:276] 0709 12:13:49.938463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.084684 (+146221us) service_pool.cc:224] Handling call
0709 12:13:50.144321 (+ 59637us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:50.956785 (+812464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.956829 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:51.014115 (+ 57286us) log.cc:844] Serialized 539 byte log entry
0709 12:13:51.667531 (+653416us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:52.604218 (+936687us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.647713 (+ 43495us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.648003 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.648116 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.648172 (+    56us) raft_consensus.cc:1685] finished
0709 12:13:52.648193 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.648390 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44155,"raft.queue_time_us":520292,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":14141,"thread_start_us":1439266,"threads_started":2,"wal-append.queue_time_us":930449}
I20260709 12:13:52.641095 25125 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:52.652249 25495 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.652387 25261 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:52.657693 25468 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:52.651821 25423 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:52.672487 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:52.679471 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:52.602630 25355 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.671020 23467 rpcz_store.cc:276] 0709 12:13:49.789783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.789934 (+   151us) service_pool.cc:224] Handling call
0709 12:13:50.626698 (+836764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.732195 (+105497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.582347 (+1850152us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54000c8850. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:52.582545 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923725568}
I20260709 12:13:52.638345 25352 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:52.672612 25467 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.000s
I20260709 12:13:52.576324 25502 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.690637 25266 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.690789 25125 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.691363 25266 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.695603 25502 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.684664 25250 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:52.683996 25495 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:52.700747 25264 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.677615 25473 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:13:52.701532 25264 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.688061 25474 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:52.704244 25264 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.657522 25494 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:13:52.674711 25503 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.706025 25503 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.707273 25418 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.708029 25418 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.685231 25423 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.684808 25468 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.714745 25472 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:52.715183 25472 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:52.659557 25432 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.715991 25472 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.716645 25468 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.717015 25472 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:52.674003 25462 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:52.692257 25125 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:52.700994 25495 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.728919 24899 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.678249 25261 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.738744 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.738804 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.739205 23394 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:52.739467 23390 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:52.740164 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.738744 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.740860 23396 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:52.741189 23391 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:52.708678 25418 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.677956 25509 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.748026 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:52.718694 25503 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:52.719357 25352 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.732841 25265 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.718114 25432 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.734095 25512 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.719902 25413 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.734563 25423 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.758132 23316 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:52.741972 25261 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.760433 24899 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.767670 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:52.779954 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:52.782196 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 218617us
I20260709 12:13:52.759243 25502 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:52.733817 25467 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.747964 25462 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.782987 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 343) took 3160 ms. Trace:
I20260709 12:13:52.791857 23387 rpcz_store.cc:276] 0709 12:13:49.618639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.678975 (+ 60336us) service_pool.cc:224] Handling call
0709 12:13:49.679285 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:49.758507 (+ 79222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:49.758538 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:49.763660 (+  5122us) log.cc:844] Serialized 26 byte log entry
0709 12:13:50.604192 (+840532us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b18002fce60. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:13:50.976559 (+372367us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:50.976600 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.779046 (+1802446us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b54002e8e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:52.779135 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.779202 (+    67us) raft_consensus.cc:1685] finished
0709 12:13:52.779223 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.779433 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":437,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":11888,"spinlock_wait_cycles":189433984,"thread_start_us":372178,"threads_started":1,"wal-append.queue_time_us":2054019}
I20260709 12:13:52.755398 25509 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:52.808595 25509 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.811690 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.811980 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.812434 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:52.812536 23470 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:52.813398 24007 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:52.814392 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:52.788158 25351 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.820724 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:52.822685 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:52.823063 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.823457 24016 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:52.826012 24011 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:52.827617 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:52.828426 23466 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:52.829759 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.832803 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:52.835192 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 181815us
I20260709 12:13:52.835314 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:52.835351 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.836077 23469 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:52.836387 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:52.764644 25512 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.841032 24013 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:52.812436 24008 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:52.802489 25502 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:52.842703 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:52.838819 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:52.888876 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46283.4us
I20260709 12:13:52.823120 25257 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.817960 25518 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:52.821237 25266 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.841919 24899 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:52.868762 25418 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.869576 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:52.953924 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 73978.3us
I20260709 12:13:52.875049 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.957124 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.957885 23469 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 1.
I20260709 12:13:52.958926 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.876039 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.876914 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.877768 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.878631 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:52.879228 25520 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:52.881448 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.889029 24011 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:52.839926 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:52.971055 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.865444 25351 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:52.979799 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.980461 24011 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:52.822655 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.889698 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:52.986717 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:52.825668 25175 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:52.951584 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:52.971457 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.987648 23470 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:52.821655 25509 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:52.973402 23982 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.195:33909 timed out after 1.663s (SENT) [suppressed 7 similar messages]
I20260709 12:13:52.842496 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:52.978186 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:13:52.981760 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:53.050115 23463 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:52.885874 25522 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:52.986408 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.891173 25250 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:52.994336 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 104763us
W20260709 12:13:53.035362 25484 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:13:53.022907 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.017477 24010 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:52.968448 25398 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:52.947075 25423 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:52.967780 25498 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:13:53.036545 25518 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:53.043956 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65347us
I20260709 12:13:52.986441 25520 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.029515 25504 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.030053 23464 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:53.019928 25503 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.019647 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.018337 25351 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:52.995543 25496 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:53.048843 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.183372 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:53.183830 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.020989 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:52.985728 25427 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.319s	user 0.013s	sys 0.000s
I20260709 12:13:53.080658 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.210976 24016 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:53.082594 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.084770 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.087464 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.089594 24899 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.211869 23470 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:53.093150 25257 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:53.094542 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.218428 24015 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:53.095732 25266 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.097690 25467 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.097796 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.110544 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:53.112494 25502 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.113147 23315 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:53.114504 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.114625 25462 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.023340 25526 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:53.030767 25521 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:53.116932 23396 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:53.041648 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:52.963591 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.044652 25408 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.138221 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1335) took 1304 ms. Trace:
I20260709 12:13:53.214993 25496 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:53.115465 25423 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:53.185768 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 212375us
W20260709 12:13:53.275399 23982 connection.cc:256] RPC call timeout handler was delayed by -1.2197129080000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 12:13:53.152499 25503 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:53.208863 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 175183us
I20260709 12:13:53.249713 25521 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.282675 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:53.125952 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.212321 24899 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.220918 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.223753 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.222263 24013 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:53.154610 25350 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.564s	user 0.012s	sys 0.003s
I20260709 12:13:53.227698 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.135984 25518 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.234380 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.236462 25400 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.238998 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.240733 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1268) took 3301 ms. Trace:
I20260709 12:13:53.241518 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.247216 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.294049 23389 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
W20260709 12:13:53.249230 25527 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:13:53.115944 25250 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
W20260709 12:13:53.227604 25532 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:13:53.143867 25504 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.257453 23395 rpcz_store.cc:276] 0709 12:13:51.833347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.833495 (+   148us) service_pool.cc:224] Handling call
0709 12:13:52.625548 (+792053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.625803 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.625842 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.625899 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:52.625943 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.626045 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.137981 (+511936us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54000a8550. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:53.138143 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820824320}
I20260709 12:13:53.312882 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.313848 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.314002 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.137282 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 466) took 3308 ms. Trace:
I20260709 12:13:53.314661 24014 rpcz_store.cc:276] 0709 12:13:49.828601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.829526 (+   925us) service_pool.cc:224] Handling call
0709 12:13:49.829844 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.993339 (+1163495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.136901 (+2143562us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400398fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:53.137192 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2582493056}
I20260709 12:13:53.316201 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.292032 25462 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:53.319314 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.260780 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.261420 25408 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.327224 23317 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:53.251681 25404 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.262533 24007 raft_consensus.cc:2410] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:53.267257 25489 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:53.269085 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 365) took 1570 ms. Trace:
I20260709 12:13:53.169199 23463 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:53.344974 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.278147 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.252017 25526 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:53.287977 24015 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:13:53.284603 23387 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:53.284716 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:13:53.349026 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 64382.2us
I20260709 12:13:53.276613 25423 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:53.289047 23312 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:53.292110 23319 rpcz_store.cc:276] 0709 12:13:49.939709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.085277 (+145568us) service_pool.cc:224] Handling call
0709 12:13:50.086415 (+  1138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.886507 (+800092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.886552 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.887152 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.565245 (+678093us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:52.163826 (+598581us) log.cc:844] Serialized 14 byte log entry
0709 12:13:52.168553 (+  4727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.240339 (+1071786us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b5400296654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:53.240391 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.240446 (+    55us) raft_consensus.cc:1685] finished
0709 12:13:53.240537 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.240675 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4683,"prepare.queue_time_us":652967,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":47,"raft.queue_time_us":639532,"raft.run_cpu_time_us":495,"raft.run_wall_time_us":121311,"spinlock_wait_cycles":111725952,"thread_start_us":1787503,"threads_started":3,"wal-append.queue_time_us":974200}
I20260709 12:13:53.309908 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.284281 25398 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.328436 25462 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.284472 25337 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.284309 25469 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.259320 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.368546 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.276015 25496 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.300787 25512 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:53.352843 25350 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:53.228827 25266 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.342512 23462 rpcz_store.cc:276] 0709 12:13:51.698774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.713391 (+ 14617us) service_pool.cc:224] Handling call
0709 12:13:51.713724 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.023471 (+309747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.023500 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.023537 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:52.023576 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.268762 (+1245186us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54001874d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:53.268823 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.269010 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711839616}
I20260709 12:13:53.353191 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.354100 23388 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:53.081398 25522 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.358284 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.393002 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.393888 24014 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:53.386863 25350 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.320544 23395 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:53.365542 25398 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.342338 25489 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:53.303663 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.343174 24008 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:53.374536 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.377244 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1606) took 3495 ms. Trace:
I20260709 12:13:53.304278 25351 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.365787 25462 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.378167 25518 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.378357 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.378628 25520 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.380436 25445 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.381067 25496 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.385140 24015 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:13:53.428313 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.356419 25257 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:53.350422 25526 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:53.392729 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.429783 23388 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:53.420614 25521 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:53.411507 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 134408us
I20260709 12:13:53.369360 23387 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:53.452765 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.370186 25519 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:13:53.455214 25519 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:53.455544 25519 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:53.456369 25519 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.413256 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.457743 23319 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:53.417150 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 103399us
I20260709 12:13:53.417224 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:53.398703 25472 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.681s	user 0.010s	sys 0.002s
I20260709 12:13:53.374655 25469 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.459839 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.372074 25337 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.421813 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.412030 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.460421 23395 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:53.490623 25503 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.491678 24015 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEARNER]: Advancing to term 5
I20260709 12:13:53.492087 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.492703 25469 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.493283 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.494385 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:53.494843 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.495857 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.495947 24899 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:53.497450 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.498121 25266 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.499008 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.500046 25496 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:53.401572 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.500419 25496 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.500790 25496 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.500878 24007 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:53.424559 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:53.502564 25490 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.525s	user 0.010s	sys 0.028s
I20260709 12:13:53.502530 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.503224 23396 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:53.504297 23396 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:53.366004 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.505714 23316 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:53.506544 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.507241 25498 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.507880 23316 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:53.429493 25257 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:53.509650 25257 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:53.509555 25148 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:53.511899 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:53.512787 23391 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:13:53.513746 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:53.514928 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:53.433470 25538 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:13:53.433912 25250 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.435027 25522 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.435362 25398 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:53.436066 24009 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:13:53.527675 24009 pending_rounds.cc:85] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 4
I20260709 12:13:53.528144 24009 pending_rounds.cc:107] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:13:53.528470 24009 raft_consensus.cc:2889] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:53.436208 25489 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.436071 24008 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:53.530383 25489 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.437204 24010 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:13:53.439659 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52264 (request call id 83) took 1212 ms. Trace:
I20260709 12:13:53.443480 23388 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:13:53.444972 25462 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.446242 25428 raft_consensus.cc:993] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:53.448604 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.453326 23387 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:13:53.527716 25261 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.003s
I20260709 12:13:53.489766 25504 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.413146 25350 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.517195 23391 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:53.520160 25528 raft_consensus.cc:2764] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:13:53.523136 25163 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:53.523317 25358 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:53.485399 25403 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:13:53.507747 25498 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.524338 25481 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.513631 25519 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:53.543907 24014 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:53.460988 23318 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:53.459312 23982 leader_election.cc:400] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7.
I20260709 12:13:53.581148 23982 leader_election.cc:403] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:53.534448 24015 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:53.548521 23393 rpcz_store.cc:276] 0709 12:13:52.284621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.284830 (+   209us) service_pool.cc:224] Handling call
0709 12:13:53.439390 (+1154560us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54002e9854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a967a735 00007ff0a96729fa 00007ff0ae63a4c9 00007ff0a7bb6c54 00007ff0a7bb6be2 00007ff0a7bb6b11 00007ff0a7bb6a8c 00007ff0a7bb5d42 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:53.439586 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617494912}
I20260709 12:13:53.550236 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.582679 23320 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:53.555397 23980 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:53.556052 25408 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.548316 25526 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.558861 25337 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.560818 23319 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:53.551103 25492 raft_consensus.cc:1081] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:13:53.575438 23316 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:53.565580 25163 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:53.576020 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 125488us
I20260709 12:13:53.551954 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.607097 23388 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:13:53.534175 23312 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:53.586298 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 127096us
I20260709 12:13:53.597576 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.578614 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.608882 23982 leader_election.cc:400] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:53.609127 23391 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:53.609305 23982 leader_election.cc:403] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:53.574913 25358 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:53.612598 23387 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:53.613353 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.614176 23391 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:53.614291 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.614869 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:53.615469 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.615662 25408 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:53.615760 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:53.616144 25408 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:53.616580 25408 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.616744 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:53.617100 25524 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:53.617493 25524 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.617609 25363 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:53.617867 25524 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:53.618000 25363 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.618395 25363 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.621958 25337 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:53.622040 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.616523 25465 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:53.625176 25363 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.625711 25408 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.626899 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.645035 23391 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:53.627737 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.629410 23388 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:53.646600 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.647177 23388 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:53.634236 23387 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:53.562041 25261 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.596169 25522 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7.
I20260709 12:13:53.654389 23393 raft_consensus.cc:2368] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:53.598574 25475 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:53.598694 25351 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.611943 25163 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.665118 24014 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:53.629626 25358 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:53.696935 23467 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 12:13:53.675791 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52650.6us
I20260709 12:13:53.694275 23388 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:53.612219 25418 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:13:53.653393 25528 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:53.718144 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.708003 25261 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:53.707823 25549 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:53.711361 25526 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.660034 25524 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.715062 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:53.715328 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.741180 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.422600 23313 rpcz_store.cc:276] 0709 12:13:49.882065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.893101 (+ 11036us) service_pool.cc:224] Handling call
0709 12:13:51.215514 (+1322413us) spinlock_profiling.cc:243] Waited 9.41 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:51.215741 (+   227us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.383003 (+1167262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.417087 (+ 34084us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:52.603053 (+185966us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.947246 (+344193us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:52.963984 (+ 16738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.970459 (+  6475us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.354473 (+384014us) raft_consensus.cc:1685] finished
0709 12:13:53.356631 (+  2158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.377169 (+ 20538us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227989,"raft.queue_time_us":761179,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":11775,"spinlock_wait_cycles":21632896,"thread_start_us":966344,"threads_started":2,"wal-append.queue_time_us":376380}
I20260709 12:13:53.722095 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.705092 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.697186 25542 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:53.656112 25148 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:13:53.698042 25351 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.730000 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.724006 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.759364 25408 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.759466 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:53.760468 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.761071 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.761739 24010 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 1.
I20260709 12:13:53.761854 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.762676 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.762890 25257 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.731765 24012 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:53.764591 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.764977 25257 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.765354 24012 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7. Re-sending same reply.
I20260709 12:13:53.738173 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.768359 23462 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:53.768151 25423 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.766528 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.769593 24012 raft_consensus.cc:2376] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5. Re-sending same reply.
I20260709 12:13:53.745813 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.658685 25363 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.741988 23314 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:53.770987 24011 raft_consensus.cc:2410] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:53.771054 25524 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.772003 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.763849 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.774399 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.759042 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.773245 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.779625 24015 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 2: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:53.765599 23464 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:53.744309 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.776646 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.758038 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.769449 23469 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:53.773360 23314 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:13:53.742895 23391 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:13:53.776754 24899 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:53.763259 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:53.788110 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:53.811463 23470 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:53.801862 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.811851 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.813213 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.815076 23312 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:53.803411 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.815711 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.816725 24015 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:53.750313 25465 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:53.817301 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.754714 25337 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:53.823211 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:53.844609 24899 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:13:53.845149 24899 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:53.845649 24899 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:53.806685 25427 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.803359 25524 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.828351 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.811463 25350 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.850078 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.806900 25418 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.854993 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.858592 25498 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.852864 25489 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.859278 25465 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.859583 25489 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.852874 25373 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.856889 25427 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.862119 25373 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.812664 25257 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:13:53.863946 25427 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.864537 25489 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.864142 25496 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.865098 25423 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.865811 23461 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:53.864163 25373 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.868252 23320 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:53.818531 25519 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:53.869992 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:53.873921 25337 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.874784 25350 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.874639 25496 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.876314 23391 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:53.884433 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.884641 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.885162 23391 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7.
I20260709 12:13:53.861511 25261 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:53.878706 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:53.857380 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 204180us
I20260709 12:13:53.880460 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.887235 23395 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:53.868129 25524 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.853732 25462 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.857067 25504 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.881975 23313 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:13:53.889904 25408 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:53.884063 25519 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.891029 25358 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.890057 25261 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:53.892637 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.893697 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.890027 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.895368 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.900826 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.867431 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.883638 25423 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:53.904469 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.877514 25559 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.916949 23982 leader_election.cc:400] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
W20260709 12:13:53.907915 25553 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:13:53.912048 23395 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:53.893699 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.914567 23387 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:53.914743 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.912631 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.925793 23466 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:53.922279 23393 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:53.922864 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.919099 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:53.911463 24011 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:53.926824 24899 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.927690 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.928786 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:53.929219 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.929822 23466 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:53.886551 25418 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.933055 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:53.935945 23395 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:13:53.912823 25398 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:53.950024 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:53.946026 25545 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:53.946293 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.955739 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.956463 24008 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:13:53.957729 25351 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:53.958847 23395 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:13:53.963128 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:53.948206 23470 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:53.948608 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:53.950189 24016 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:53.942874 25358 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.903220 25504 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:53.951413 23466 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:53.941601 24899 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:53.915231 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.967907 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:53.977136 23470 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:53.968215 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.979663 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.979806 23387 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:13:53.975963 25509 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:53.982139 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1342) took 2127 ms (client timeout 1979 ms). Trace:
I20260709 12:13:53.982254 25509 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:53.982561 23392 rpcz_store.cc:269] 0709 12:13:51.854923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.981891 (+2126968us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:53.982044 (+   153us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:53.983716 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1350) took 1445 ms. Trace:
I20260709 12:13:53.930058 25522 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:13:53.984110 23394 rpcz_store.cc:276] 0709 12:13:52.538159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.739960 (+201801us) service_pool.cc:224] Handling call
0709 12:13:52.740253 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.747980 (+  7727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.748029 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:52.755762 (+  7733us) log.cc:844] Serialized 26 byte log entry
0709 12:13:53.852503 (+1096741us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b1800418c60. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:13:53.912140 (+ 59637us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:53.912177 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.912237 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.983426 (+ 71189us) raft_consensus.cc:1685] finished
0709 12:13:53.983445 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.983658 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3406,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5359,"spinlock_wait_cycles":6667520,"thread_start_us":58890,"threads_started":1,"wal-append.queue_time_us":1190907}
I20260709 12:13:53.986230 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1625) took 1148 ms. Trace:
I20260709 12:13:53.986598 23321 rpcz_store.cc:276] 0709 12:13:52.837602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.837731 (+   129us) service_pool.cc:224] Handling call
0709 12:13:53.558321 (+720590us) spinlock_profiling.cc:243] Waited 894 us on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:53.558511 (+   190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.558887 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.558920 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:53.559416 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 12:13:53.600424 (+ 41008us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:13:53.600466 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.600527 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:53.985957 (+385430us) raft_consensus.cc:1685] finished
0709 12:13:53.985974 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:53.986175 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1719,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":11406,"spinlock_wait_cycles":2055936,"thread_start_us":40592,"threads_started":1,"wal-append.queue_time_us":192626}
I20260709 12:13:53.987638 25504 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:53.989029 23392 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:13:53.981572 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57175us
I20260709 12:13:53.949256 25261 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:53.995266 23466 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:53.991554 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:53.940025 25555 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:13:54.003383 25418 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:53.988454 25475 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:54.004405 25475 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:54.005043 25519 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.005514 25475 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.006560 25524 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:54.009294 25163 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.011610 25163 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:54.024336 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1271) took 1200 ms. Trace:
I20260709 12:13:54.026554 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1632) took 1157 ms. Trace:
I20260709 12:13:54.027048 23317 rpcz_store.cc:276] 0709 12:13:52.869366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.087158 (+217792us) service_pool.cc:224] Handling call
0709 12:13:54.026329 (+939171us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54003af054. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.026472 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264464128}
I20260709 12:13:54.024811 23315 rpcz_store.cc:276] 0709 12:13:52.823566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.823681 (+   115us) service_pool.cc:224] Handling call
0709 12:13:52.834452 (+ 10771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.229707 (+395255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.024027 (+794320us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400240ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:54.024265 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640165248}
I20260709 12:13:54.033263 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.040918 25163 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:54.043861 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.035714 25358 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.049618 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:54.051009 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.065213 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.066912 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:54.069060 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:54.046379 25363 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:54.049407 25358 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.067953 25148 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:54.066196 25569 raft_consensus.cc:2764] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:54.069847 25561 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:54.071099 25464 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:54.074947 25408 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:13:54.078851 25403 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.079600 25358 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:54.080699 25534 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.753s	user 0.004s	sys 0.000s
I20260709 12:13:54.091121 25464 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:54.141773 25464 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.148844 24011 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:54.115357 25408 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:54.194064 25559 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:54.196195 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 124526us
I20260709 12:13:54.205180 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:54.231257 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.348428 25569 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:54.349009 25569 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.349468 25569 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:54.363788 25568 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
W20260709 12:13:54.365803 25570 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:13:54.368309 25408 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.188968 24008 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:13:54.373484 25355 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:54.390321 25571 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:13:54.446982 24007 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:54.470496 25569 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.516806 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1643) took 1240 ms. Trace:
I20260709 12:13:54.517400 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.518643 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.520737 25261 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:54.483428 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 275162us
I20260709 12:13:54.524567 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.481159 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1634) took 1606 ms. Trace:
I20260709 12:13:54.525215 23316 rpcz_store.cc:276] 0709 12:13:52.874817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.509391 (+634574us) service_pool.cc:224] Handling call
0709 12:13:53.518645 (+  9254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.599069 (+ 80424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.480741 (+881672us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54001a7cd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:54.481055 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130036480}
I20260709 12:13:54.526517 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:54.526993 25579 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:54.527585 25579 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:54.528236 25561 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:54.528620 25579 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.529019 25148 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:54.530157 25355 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.530540 25355 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.530879 25355 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.534854 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:54.535909 25475 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:13:54.537349 25475 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:54.537809 25475 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.538232 25475 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:54.555066 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1627) took 1687 ms (client timeout 2071 ms). Trace:
W20260709 12:13:54.555570 23318 rpcz_store.cc:269] 0709 12:13:52.868027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.258692 (+390665us) service_pool.cc:224] Handling call
0709 12:13:54.554865 (+1296173us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54002760d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.554976 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260877696}
I20260709 12:13:54.557663 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:54.564077 25575 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.363s	user 0.004s	sys 0.007s
I20260709 12:13:54.570168 25355 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.495447 25509 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:54.496816 23469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1594) took 1625 ms (client timeout 2027 ms). Trace:
W20260709 12:13:54.574471 23469 rpcz_store.cc:269] 0709 12:13:52.871548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.958643 (+ 87095us) service_pool.cc:224] Handling call
0709 12:13:54.496600 (+1537957us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b54001072d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.496747 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862061440}
I20260709 12:13:54.499380 25358 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.500644 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1596) took 1628 ms. Trace:
I20260709 12:13:54.576287 23465 rpcz_store.cc:276] 0709 12:13:52.872121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.986211 (+114090us) service_pool.cc:224] Handling call
0709 12:13:54.500463 (+1514252us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b540044af54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.500588 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149531904}
I20260709 12:13:54.578063 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 348) took 1060 ms. Trace:
I20260709 12:13:54.578547 23388 rpcz_store.cc:276] 0709 12:13:53.517229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.658199 (+140970us) service_pool.cc:224] Handling call
0709 12:13:53.658471 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.698742 (+ 40271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.577810 (+879068us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b54001964d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:54.577994 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81905920}
I20260709 12:13:54.579919 25358 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.508277 23218 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 340) took 7449 ms. Trace:
I20260709 12:13:54.520382 25513 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:54.580688 25513 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:13:54.581132 25513 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:54.582023 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:54.584403 25500 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:54.584957 25500 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:54.520134 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 22fc2096e78f47158bbb4bae3a474e2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6253b834ffb744579b46fcede2fc0e91 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:13:54.586014 23206 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66172.3us
I20260709 12:13:54.517231 23320 rpcz_store.cc:276] 0709 12:13:53.276831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.583660 (+306829us) service_pool.cc:224] Handling call
0709 12:13:53.584010 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.868818 (+284808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.516507 (+647689us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400388454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:54.516739 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652900608}
I20260709 12:13:54.519500 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.587394 25526 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:54.587781 25526 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.588136 25526 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.591288 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.592540 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.593128 25468 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:54.593688 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.593901 25468 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:54.594614 25468 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:54.594808 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:54.595325 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 75894.9us
I20260709 12:13:54.596388 25559 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:54.596822 25559 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:54.597198 25559 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.597299 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.598260 25398 raft_consensus.cc:2764] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:13:54.490127 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1579) took 1701 ms. Trace:
I20260709 12:13:54.599318 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.585394 25500 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:54.598980 23467 rpcz_store.cc:276] 0709 12:13:52.788709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.794481 (+  5772us) service_pool.cc:224] Handling call
0709 12:13:52.794944 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.697384 (+902440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.489863 (+792479us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b5400225bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:54.490059 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2073877888}
W20260709 12:13:54.597384 24013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 638) took 1752 ms (client timeout 2166 ms). Trace:
W20260709 12:13:54.601011 24013 rpcz_store.cc:269] 0709 12:13:52.844900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.845088 (+   188us) service_pool.cc:224] Handling call
0709 12:13:54.597222 (+1752134us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400775e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.597328 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":616920448}
I20260709 12:13:54.602725 25403 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:54.603194 25403 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.604784 23465 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:54.606158 25427 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:54.611270 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.612182 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.614437 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1629) took 1745 ms. Trace:
I20260709 12:13:54.614811 23319 rpcz_store.cc:276] 0709 12:13:52.868609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.494140 (+625531us) service_pool.cc:224] Handling call
0709 12:13:54.614240 (+1120100us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b54000ec254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:54.614380 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150731520}
I20260709 12:13:54.618253 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.621201 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.623350 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.623704 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.629092 23467 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:13:54.480249 25464 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.630194 25559 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.631103 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.632839 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.648506 24010 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:54.648900 25559 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.634717 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:54.636742 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.636914 23387 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52266
W20260709 12:13:54.650245 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.650405 23387 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:54.650336 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.650960 23387 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:54.653306 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.637310 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.638082 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:54.660296 23387 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.662479 23387 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.662719 25464 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:54.662731 25398 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:13:54.664026 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 64777.2us
I20260709 12:13:54.664783 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.665671 23217 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.667316 23217 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.669803 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.548879 25475 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.670552 23395 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:54.672052 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.673960 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.674309 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.675086 23395 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:54.675246 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:54.677412 24010 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
W20260709 12:13:54.677800 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.677978 24014 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:54.678303 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.679034 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:54.679489 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:54.679677 24013 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:54.680088 23319 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:54.682149 25398 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:54.682547 25398 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.682873 25398 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.683564 25358 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:54.639991 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.642122 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.642503 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:54.684975 23217 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.676156 23218 rpcz_store.cc:276] 0709 12:13:47.058761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.058919 (+   158us) service_pool.cc:224] Handling call
0709 12:13:54.508207 (+7449288us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:50.216996 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:50.399937 (+182941us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:50.399990 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:13:50.400017 (+    27us) tablet.cc:662] Decoding operations
0709 12:13:50.400569 (+   552us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:50.400674 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:50.400715 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:13:50.401096 (+   381us) tablet.cc:701] Row locks acquired
0709 12:13:50.401121 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:13:50.401305 (+   184us) write_op.cc:270] Start()
0709 12:13:50.401473 (+   168us) write_op.cc:276] Timestamp: P: 1783599230401248 usec, L: 0
0709 12:13:50.401504 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:13:50.402353 (+   849us) log.cc:844] Serialized 930 byte log entry
0709 12:13:52.217507 (+1815154us) op_driver.cc:464] REPLICATION: finished
0709 12:13:52.786900 (+569393us) write_op.cc:301] APPLY: starting
0709 12:13:52.790360 (+  3460us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:52.813980 (+ 23620us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:52.823019 (+  9039us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:52.840512 (+ 17493us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:52.890553 (+ 50041us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:13:52.890618 (+    65us) write_op.cc:312] APPLY: finished
0709 12:13:52.950930 (+ 60312us) log.cc:844] Serialized 29 byte log entry
0709 12:13:52.951151 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:52.960623 (+  9472us) write_op.cc:454] Released schema lock
0709 12:13:52.968646 (+  8023us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:53.080069 (+111423us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:53.086649 (+  6580us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":379952,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":543394,"apply.run_cpu_time_us":6792,"apply.run_wall_time_us":261556,"num_ops":2,"prepare.queue_time_us":184937,"prepare.run_cpu_time_us":43327,"prepare.run_wall_time_us":1407713,"raft.queue_time_us":401726,"raft.run_cpu_time_us":120466,"raft.run_wall_time_us":361904,"replication_time_us":1815778,"thread_start_us":1735977,"threads_started":4,"wal-append.queue_time_us":1411184,"wal-append.run_cpu_time_us":19190,"wal-append.run_wall_time_us":2267830}]]}
I20260709 12:13:54.690630 25398 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.692822 25513 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:54.693830 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:54.639019 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:54.572152 25261 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:13:54.649279 25526 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:54.694993 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.674297 25475 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.724710 24012 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:13:54.740646 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:54.741354 23315 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:54.697824 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 155815us
I20260709 12:13:54.705189 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.744777 25569 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:54.744997 23465 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:54.706777 24013 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:13:54.707554 23217 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.724289 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:54.727931 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.748520 23469 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:54.749143 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:54.748577 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 5, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 5 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.749715 23321 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:54.750698 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:54.750466 23217 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 4, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.728633 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.731022 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.734829 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:54.660826 25403 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.696614 25500 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.735311 25408 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:54.747192 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:54.772856 23321 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:54.734905 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:54.776005 23318 raft_consensus.cc:2410] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:13:54.776232 23465 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:54.696070 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:54.763082 24014 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:54.765403 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.761603 23217 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:54.812399 23315 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:13:54.812721 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 114878us
I20260709 12:13:54.770483 23316 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:54.849247 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.788481 23469 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:54.853389 23217 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 6, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 6 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:13:54.858022 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 92738.3us
I20260709 12:13:54.860145 23982 leader_election.cc:400] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7.
I20260709 12:13:54.861130 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:54.861853 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:54.862723 23318 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:54.855526 23217 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 2, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:54.781926 25175 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:54.844141 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:54.808123 25408 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:54.770780 25445 consensus_queue.cc:1059] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:13:54.809742 25264 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:13:54.829674 25089 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
W20260709 12:13:55.015971 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 259634us
W20260709 12:13:55.378237 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.020s	sys 0.013s
W20260709 12:13:55.390203 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.021s	sys 0.013s
I20260709 12:13:55.390467 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:13:55.391412 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 547327us
I20260709 12:13:54.770905 25509 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:13:55.022696 23469 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:13:54.880683 23217 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.878512 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:55.403376 23217 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:55.404034 23318 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:55.029789 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 159831us
I20260709 12:13:55.406374 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:54.773576 25355 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:55.297747 23434 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.001s	sys 0.025s
W20260709 12:13:55.395043 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 545647us
W20260709 12:13:55.444896 23434 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.003s	sys 0.026s
W20260709 12:13:55.445254 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 705536us
W20260709 12:13:55.445793 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.445668 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:55.151180 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.858054 25593 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:54.982045 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:54.868784 24008 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:55.240968 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:55.245237 25148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.002s	sys 0.018s
W20260709 12:13:55.411792 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.416734 23465 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:55.448921 23468 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:54.876222 25560 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:54.864526 25475 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.022292 25261 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:13:55.450647 23316 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:55.404894 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:55.021273 25403 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:55.228852 25464 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:55.452416 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:55.266259 25500 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:55.446089 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:55.473194 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:55.475591 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.809597 25266 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.630s
I20260709 12:13:55.400228 25509 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.410673 25569 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:13:55.208340 23461 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:13:55.407866 25363 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:55.470400 23217 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 3, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:54.868273 24010 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:55.411290 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:54.907696 25559 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.554s
I20260709 12:13:55.451853 25514 raft_consensus.cc:2764] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:13:55.483253 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.003s	sys 0.030s
I20260709 12:13:55.463400 25355 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.477846 25523 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:55.447059 25468 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:13:55.473201 25464 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:55.482300 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:55.463616 23313 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:55.477360 25526 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:55.479611 25509 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:55.484844 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.003s	sys 0.030s
I20260709 12:13:55.473510 25398 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:55.483537 25264 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.501252 23316 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:55.490937 23320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.005s	sys 0.030s
W20260709 12:13:55.546303 23320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.005s	sys 0.030s
I20260709 12:13:55.259348 25596 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.206s
I20260709 12:13:55.504065 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:55.461577 23318 raft_consensus.cc:2410] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:13:55.485548 23360 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.037s
W20260709 12:13:55.547600 23360 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.010s	sys 0.037s
I20260709 12:13:55.489758 25591 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:13:55.491681 23313 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:55.491685 25363 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:55.557935 25264 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:55.559669 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:55.527585 25523 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:13:55.531432 23319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.006s	sys 0.007s
W20260709 12:13:55.566650 23319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.007s	sys 0.008s
I20260709 12:13:55.461225 25593 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:55.500830 25559 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:55.569697 25559 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:55.570925 25559 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:55.572654 25559 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:55.569120 25593 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:55.532305 25513 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:55.548013 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 594675us
I20260709 12:13:55.490267 25355 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:55.552345 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 48368.9us
I20260709 12:13:55.499133 25526 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:55.547919 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:55.599819 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:55.600873 23319 raft_consensus.cc:2410] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:13:55.593885 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 203531us
I20260709 12:13:55.618949 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:55.598021 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:55.620834 23316 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.615701 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:55.593652 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:55.602723 23468 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:13:55.621265 23434 connection.cc:256] RPC call timeout handler was delayed by -1.6497750369999267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3249s for a response
I20260709 12:13:55.626106 25513 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.627707 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:55.560997 25603 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:55.629597 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 177266us
I20260709 12:13:55.592963 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:55.641964 23320 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.636710 23316 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:13:55.643981 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 204873us
I20260709 12:13:55.641999 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:55.652413 23316 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:13:55.652861 23316 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:55.654058 23316 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:55.589700 25601 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.589092 25607 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:55.704972 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 54419.1us
I20260709 12:13:55.707404 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:55.731676 23316 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:55.555270 25464 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:55.734642 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 141101us
I20260709 12:13:55.616351 25599 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:13:55.725910 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 75320.8us
I20260709 12:13:55.773860 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.774722 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:55.775950 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.777884 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.639185 25408 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:55.718781 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.779495 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:55.636459 25526 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:55.630687 25089 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 12:13:55.714749 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:55.837165 23316 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:55.710872 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:55.848472 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73972.4us
I20260709 12:13:55.843869 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.652175 25513 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:55.852006 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.975283 23467 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:55.823182 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:55.862353 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 243480us
W20260709 12:13:55.718467 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68425.1us
I20260709 12:13:55.976301 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.948267 23314 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:13:55.710125 25605 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:55.709820 25593 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:55.771099 25500 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:55.837193 23320 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:55.968839 23316 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:55.654317 25610 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:55.861761 25588 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:13:55.769459 25599 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:55.653813 25603 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:55.770478 25601 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:55.729178 23313 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:55.976801 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:13:56.134178 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 157337us
W20260709 12:13:56.000645 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.260286 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 476) took 1175 ms. Trace:
I20260709 12:13:55.861618 25621 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:13:56.260608 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.808552 25579 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:56.261313 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:56.262857 23468 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:13:56.301653 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.268157 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1665) took 1257 ms. Trace:
I20260709 12:13:56.309245 23319 rpcz_store.cc:276] 0709 12:13:55.010689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.622664 (+611975us) service_pool.cc:224] Handling call
0709 12:13:55.622976 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.623211 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:13:55.623924 (+   713us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.267734 (+643810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.267803 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:56.267880 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.267962 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.268089 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119919,"prepare.queue_time_us":228,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"thread_start_us":203233,"threads_started":1}
I20260709 12:13:55.953126 23470 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:56.130882 25514 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.185s
I20260709 12:13:56.071907 23461 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:13:55.977147 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.266127 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 499840us
I20260709 12:13:56.098254 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.267405 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.281118 23320 rpcz_store.cc:276] 0709 12:13:55.085325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.592624 (+507299us) service_pool.cc:224] Handling call
0709 12:13:56.260211 (+667587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.281618 23467 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:56.283325 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 465) took 1201 ms. Trace:
I20260709 12:13:56.286883 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.771445 25607 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.356506 23314 rpcz_store.cc:276] 0709 12:13:55.082032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.936941 (+854909us) service_pool.cc:224] Handling call
0709 12:13:55.947956 (+ 11015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.282402 (+334446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.283271 (+   869us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:56.284873 25606 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.752s	user 0.000s	sys 0.001s
I20260709 12:13:56.357234 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.269510 25500 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:56.357946 23466 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:13:56.358114 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.358022 25607 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.358613 25523 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:56.284322 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 466) took 1202 ms. Trace:
I20260709 12:13:56.359102 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.359171 23316 rpcz_store.cc:276] 0709 12:13:55.082322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.968215 (+885893us) service_pool.cc:224] Handling call
0709 12:13:55.968619 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.284021 (+315402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.284254 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.358120 25500 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.359899 23466 raft_consensus.cc:2410] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:13:56.359172 25523 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:56.361028 23466 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:55.968127 24010 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.359812 23467 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:13:56.362241 25523 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:56.362555 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 467) took 1280 ms. Trace:
I20260709 12:13:56.363345 23319 rpcz_store.cc:276] 0709 12:13:55.082601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.361839 (+1279238us) service_pool.cc:224] Handling call
0709 12:13:56.362112 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.362276 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.362301 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.362338 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.362376 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.362423 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.362508 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.364120 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:56.334195 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.330380 25610 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:56.278144 25596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.026s	sys 0.017s
W20260709 12:13:56.041081 23315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.029s	sys 0.098s
I20260709 12:13:56.358626 23469 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:56.209174 25559 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.352289 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.130349 25560 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.096987 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:55.862498 25622 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.624s
W20260709 12:13:56.450917 25358 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.020s	sys 0.026s
W20260709 12:13:56.484192 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.013s	sys 0.036s
W20260709 12:13:56.534881 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.013s	sys 0.037s
I20260709 12:13:56.485476 23462 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 2. Ops: []
I20260709 12:13:56.484922 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:56.485087 25608 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.780s	user 0.001s	sys 0.000s
I20260709 12:13:56.489934 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.511813 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:56.545192 23469 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:56.034624 25398 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.464s
I20260709 12:13:56.131361 25569 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:56.301016 25464 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:55.862675 25363 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:56.298378 25579 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:56.565574 25579 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:56.567126 25579 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:56.573925 25624 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
W20260709 12:13:56.459254 25615 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.531s	user 0.000s	sys 0.001s
I20260709 12:13:56.238150 25089 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.484810 25596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.028s	sys 0.020s
I20260709 12:13:56.550622 23468 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:56.334501 25605 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:13:56.558775 25175 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.119s	user 0.007s	sys 0.003s
W20260709 12:13:56.534746 25358 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.024s	sys 0.031s
I20260709 12:13:56.285305 25621 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:56.301278 25593 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.240s
I20260709 12:13:56.228945 25618 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.418s
I20260709 12:13:56.274677 25526 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:56.578960 25475 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.016s	sys 0.025s
I20260709 12:13:56.523947 25617 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:56.524217 25355 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.294452 25599 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:56.526669 25559 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:56.607462 25612 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.863s	user 0.001s	sys 0.000s
I20260709 12:13:56.606376 25403 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.493563 25468 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:13:56.549768 25607 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.601279 25614 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.464s	user 0.001s	sys 0.001s
I20260709 12:13:56.526104 25560 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.599336 25626 raft_consensus.cc:2764] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:13:56.602651 25632 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:13:56.631129 25475 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.017s	sys 0.025s
W20260709 12:13:56.640767 25592 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.019s	sys 0.065s
I20260709 12:13:56.644661 25632 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:56.645043 25640 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:56.645507 25610 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:56.645635 25640 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.646103 25621 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.645870 25632 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.582321 25569 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.634887 25617 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:56.647224 25621 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:13:56.657723 25619 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:13:56.659576 25613 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:13:56.661906 25617 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.641489 25607 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.667740 25607 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.652324 25637 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:56.670178 25637 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.173794 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:56.721699 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.722493 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.333738 23313 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:13:56.486893 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 122868us
I20260709 12:13:56.727012 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1655) took 1978 ms. Trace:
W20260709 12:13:56.485330 23315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.037s	sys 0.098s
I20260709 12:13:56.450372 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 468) took 1367 ms. Trace:
I20260709 12:13:56.730315 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1663) took 1736 ms. Trace:
I20260709 12:13:56.731280 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:56.730861 23315 rpcz_store.cc:276] 0709 12:13:54.993400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.993615 (+   215us) service_pool.cc:224] Handling call
0709 12:13:54.994027 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.994302 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:13:55.483342 (+489040us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.729341 (+1245999us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.729462 (+   121us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:56.729648 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.729833 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.730107 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57519,"prepare.queue_time_us":67350,"prepare.run_cpu_time_us":559,"prepare.run_wall_time_us":25837,"thread_start_us":1733852,"threads_started":2,"wal-append.queue_time_us":1121867}
I20260709 12:13:56.734208 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1653) took 1986 ms. Trace:
I20260709 12:13:56.734715 23317 rpcz_store.cc:276] 0709 12:13:54.747913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.751624 (+  3711us) service_pool.cc:224] Handling call
0709 12:13:54.805735 (+ 54111us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.280509 (+474774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.280556 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:55.409964 (+129408us) log.cc:844] Serialized 52 byte log entry
0709 12:13:55.948692 (+538728us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:55.948754 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.948850 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.733653 (+784803us) raft_consensus.cc:1685] finished
0709 12:13:56.733701 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.734030 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35997,"raft.queue_time_us":705394,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":25370,"thread_start_us":905800,"threads_started":2,"wal-append.queue_time_us":1084237}
W20260709 12:13:56.484438 23361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.022s	sys 0.049s
W20260709 12:13:56.736979 23361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.023s	sys 0.050s
I20260709 12:13:56.730369 23319 rpcz_store.cc:276] 0709 12:13:55.082897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.364348 (+1281451us) service_pool.cc:224] Handling call
0709 12:13:56.449775 (+ 85427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.449979 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.450005 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.450076 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:56.450128 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.450183 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.450312 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.737259 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1658) took 1970 ms. Trace:
W20260709 12:13:56.599836 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 452492us
W20260709 12:13:56.777786 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.006s	sys 0.005s
I20260709 12:13:56.727766 23312 rpcz_store.cc:276] 0709 12:13:54.748469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.750385 (+  1916us) service_pool.cc:224] Handling call
0709 12:13:54.814327 (+ 63942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.814562 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.814586 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.855182 (+ 40596us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.493586 (+638404us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.493631 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.493699 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.726197 (+1232498us) raft_consensus.cc:1685] finished
0709 12:13:56.726250 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.726706 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":510962,"threads_started":1,"wal-append.queue_time_us":708359}
W20260709 12:13:56.644660 25592 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.019s	sys 0.065s
I20260709 12:13:56.600690 25175 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:13:56.790359 25655 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:13:56.801887 25644 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:56.817683 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.006s	sys 0.005s
I20260709 12:13:56.737811 23321 rpcz_store.cc:276] 0709 12:13:54.767206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.773746 (+  6540us) service_pool.cc:224] Handling call
0709 12:13:54.774490 (+   744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.274911 (+500421us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:13:55.274952 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.301514 (+ 26562us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.511845 (+210331us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:55.512033 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.512170 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.736756 (+1224586us) raft_consensus.cc:1685] finished
0709 12:13:56.736806 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.737070 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49216,"raft.queue_time_us":721998,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":30199,"thread_start_us":704912,"threads_started":2,"wal-append.queue_time_us":823564}
W20260709 12:13:56.797298 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60937us
I20260709 12:13:56.596949 25089 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:56.834848 23981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.008s	sys 0.002s
W20260709 12:13:56.835562 23981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.008s	sys 0.002s
W20260709 12:13:56.836462 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.542774 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:13:56.842007 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2194) took 1070 ms. Trace:
I20260709 12:13:56.842586 23464 rpcz_store.cc:276] 0709 12:13:55.771020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.781452 (+ 10432us) service_pool.cc:224] Handling call
0709 12:13:56.841726 (+1060274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.843809 25560 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:56.655144 25605 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:56.849749 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.010s
W20260709 12:13:56.850404 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.000s	sys 0.010s
W20260709 12:13:56.850835 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 568893us
I20260709 12:13:56.851035 25603 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:56.854152 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.636688 25468 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:56.870244 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 478) took 1481 ms. Trace:
I20260709 12:13:56.870877 23321 rpcz_store.cc:276] 0709 12:13:55.389300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.856192 (+1466892us) service_pool.cc:224] Handling call
0709 12:13:56.864321 (+  8129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.869753 (+  5432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.869786 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.869860 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.869923 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.869984 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.870126 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.874332 25651 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:56.875393 25651 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.878885 25635 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:56.882601 23321 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:13:56.884897 25089 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.885406 25644 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.879582 25635 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:56.853169 25403 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.883121 23321 pending_rounds.cc:85] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
W20260709 12:13:56.915756 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55137.3us
I20260709 12:13:56.903474 25261 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:13:56.922634 25175 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.900314 25646 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:56.923508 25175 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:56.924661 25513 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:56.905932 25623 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 12:13:56.908278 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 477) took 1808 ms. Trace:
I20260709 12:13:56.928510 23319 rpcz_store.cc:276] 0709 12:13:55.099461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.838910 (+1739449us) service_pool.cc:224] Handling call
0709 12:13:56.906975 (+ 68065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.907661 (+   686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.907697 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.907780 (+    83us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.907846 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.908037 (+   191us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.908175 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.914575 23321 pending_rounds.cc:107] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:13:56.927106 25623 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:56.945577 23321 raft_consensus.cc:2889] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.904001 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:56.899441 25635 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:56.889386 25644 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:56.913162 25636 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:56.894227 25601 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:56.897855 25569 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.911216 25403 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:56.911790 25616 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:56.927357 25637 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:56.905443 25475 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.984356 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:57.001844 23313 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1667) took 1920 ms (client timeout 1504 ms). Trace:
W20260709 12:13:56.974138 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.002324 25611 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.065s	user 0.001s	sys 0.005s
I20260709 12:13:56.978008 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1629) took 1130 ms. Trace:
I20260709 12:13:57.002983 23467 rpcz_store.cc:276] 0709 12:13:55.847974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.362044 (+514070us) service_pool.cc:224] Handling call
0709 12:13:56.362339 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.788627 (+426288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.788704 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.791765 (+  3061us) log.cc:844] Serialized 26 byte log entry
0709 12:13:56.821760 (+ 29995us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:56.821811 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.833835 (+ 12024us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.977515 (+143680us) raft_consensus.cc:1685] finished
0709 12:13:56.977540 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.977770 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":282156,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":454860,"threads_started":2,"wal-append.queue_time_us":146274}
I20260709 12:13:56.957033 25623 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:57.005524 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:57.006417 25559 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:57.006544 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53982.3us
I20260709 12:13:57.006896 25610 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:57.007644 25610 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:57.013491 23392 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:56.981320 25148 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.039340 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.039094 25148 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.040530 23317 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:57.044024 25640 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:57.046824 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61006.6us
I20260709 12:13:57.009277 25610 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.002346 23313 rpcz_store.cc:269] 0709 12:13:55.080969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.116487 (+ 35518us) service_pool.cc:224] Handling call
0709 12:13:57.001681 (+1885194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.981529 25632 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.982008 25264 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:13:56.982306 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 956251us
W20260709 12:13:56.984284 23434 connection.cc:256] RPC call timeout handler was delayed by -1.7817425340001591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23220000000000002s for a response
I20260709 12:13:56.989764 25645 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:13:56.990700 25649 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:56.965741 25636 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:56.993590 25542 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:13:57.055030 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 151298us
I20260709 12:13:56.995009 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1670) took 1405 ms. Trace:
I20260709 12:13:56.997812 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.000553 25560 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:56.969269 25175 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.001542 25475 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.971855 25616 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:13:57.061761 25089 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.062595 25089 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.062674 25616 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.064519 25616 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:56.980520 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 479) took 1565 ms. Trace:
I20260709 12:13:57.009879 25623 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.065893 25148 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.066807 25355 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.067634 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:57.067576 25355 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:56.994127 25651 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.068995 23392 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
W20260709 12:13:57.070540 25579 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.015s
I20260709 12:13:57.065649 23321 rpcz_store.cc:276] 0709 12:13:55.414975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.872182 (+1457207us) service_pool.cc:224] Handling call
0709 12:13:56.872806 (+   624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.980191 (+107385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.980440 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:57.073226 25648 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:13:57.073976 25648 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:57.074894 25475 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.075073 25648 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.075587 25560 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.075937 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.061050 25636 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.077631 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.078148 25560 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.070570 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:57.079439 23395 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:57.081914 23314 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:57.085141 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:57.065353 25089 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.085995 24014 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:57.051062 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43585.9us
W20260709 12:13:57.059005 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53319.8us
I20260709 12:13:57.055625 23319 rpcz_store.cc:276] 0709 12:13:55.589995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.959042 (+1369047us) service_pool.cc:224] Handling call
0709 12:13:56.993858 (+ 34816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.994241 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.994270 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.994660 (+   390us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:13:56.994725 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.994783 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.994920 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:57.059456 25632 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.061110 25509 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.522s	user 0.013s	sys 0.009s
I20260709 12:13:57.066890 25623 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:57.059897 25542 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:13:57.053324 25264 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:57.092666 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44813.1us
I20260709 12:13:56.895722 25468 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.053598 25649 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:57.077641 25636 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.106318 25599 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.108407 25660 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.108954 25599 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.109130 25660 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.113368 23461 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.002s
I20260709 12:13:57.113790 25653 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.114250 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:57.114856 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:57.114655 25653 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.115674 24014 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:57.092137 25542 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:57.113919 25175 raft_consensus.cc:1064] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:13:57.104319 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:57.131336 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.124830 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 469) took 2036 ms. Trace:
I20260709 12:13:57.131846 23315 rpcz_store.cc:276] 0709 12:13:55.083189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.820350 (+1737161us) service_pool.cc:224] Handling call
0709 12:13:56.863097 (+ 42747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.911913 (+ 48816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.911983 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.912746 (+   763us) log.cc:844] Serialized 26 byte log entry
0709 12:13:56.971247 (+ 58501us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.971568 (+   321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.971644 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.119146 (+147502us) raft_consensus.cc:1685] finished
0709 12:13:57.119168 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.119314 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2045,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":34087,"spinlock_wait_cycles":1679744,"thread_start_us":57990,"threads_started":1,"wal-append.queue_time_us":202609}
I20260709 12:13:57.128194 25660 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.141773 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:57.142590 24014 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:57.143114 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:13:57.144032 24010 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.694s	user 0.009s	sys 0.005s
I20260709 12:13:57.144286 25640 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:13:57.145737 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.146605 23468 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 2.
I20260709 12:13:57.146744 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:57.147212 25640 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:57.144554 24010 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:57.148739 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 663) took 1554 ms. Trace:
I20260709 12:13:57.142748 25175 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.099279 25560 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:13:57.114106 23461 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:57.149175 24010 rpcz_store.cc:276] 0709 12:13:55.593800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.600095 (+  6295us) service_pool.cc:224] Handling call
0709 12:13:57.148609 (+1548514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:57.156105 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.097692 25509 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.156407 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:57.161543 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66223.2us
I20260709 12:13:57.156832 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1672) took 1566 ms. Trace:
I20260709 12:13:57.162359 23313 rpcz_store.cc:276] 0709 12:13:55.590677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.054097 (+1463420us) service_pool.cc:224] Handling call
0709 12:13:57.156161 (+102064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:57.156505 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.156530 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:57.156585 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:57.156629 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.156685 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.156781 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1408}
I20260709 12:13:57.157234 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:57.163309 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67190.1us
I20260709 12:13:57.173271 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.180851 23466 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:57.182255 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:57.183466 24010 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:13:57.184576 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 471) took 2100 ms. Trace:
W20260709 12:13:57.185591 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1619) took 1592 ms (client timeout 1719 ms). Trace:
I20260709 12:13:57.105454 25644 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:13:57.118809 25542 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.257478 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 82927.9us
I20260709 12:13:57.184966 23316 rpcz_store.cc:276] 0709 12:13:55.083795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.793277 (+1709482us) service_pool.cc:224] Handling call
0709 12:13:56.794239 (+   962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.794913 (+   674us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:56.962939 (+168026us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.184124 (+221185us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:57.184167 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:57.184234 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.184302 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.184408 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21438,"prepare.queue_time_us":306578,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":231758,"threads_started":2}
W20260709 12:13:57.227560 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.256307 23394 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:13:57.162484 25653 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:13:57.274012 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86753.7us
W20260709 12:13:57.285012 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.286326 23318 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:13:57.284071 23314 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:57.286320 23468 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:57.185767 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:57.256208 25500 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.810s	user 0.013s	sys 0.004s
I20260709 12:13:57.291999 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 475) took 2207 ms. Trace:
I20260709 12:13:57.186264 25633 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:13:57.161356 25264 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.299695 23464 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:57.300341 25264 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:57.300752 25661 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.301337 25661 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.303694 25661 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.303974 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.256428 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 666) took 1553 ms. Trace:
I20260709 12:13:57.186674 25617 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.186887 25621 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:57.186260 25355 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.186112 25660 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:13:57.306375 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 192270us
W20260709 12:13:57.605893 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.313545 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.316890 23469 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:13:57.411706 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.587888 24016 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:57.331136 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.315203 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:57.406517 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.564896 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57384.2us
I20260709 12:13:57.617700 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:57.358846 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.407573 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:57.617590 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:13:57.617818 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.450089 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80569.9us
W20260709 12:13:57.262691 23461 rpcz_store.cc:269] 0709 12:13:55.593397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.598709 (+  5312us) service_pool.cc:224] Handling call
0709 12:13:57.185491 (+1586782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:57.295071 25148 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.294770 25475 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:13:57.624442 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.256884 25649 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.308558 25408 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.372687 25645 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.313921 25632 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.358872 25264 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.538412 25667 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 12:13:57.595685 25670 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
W20260709 12:13:57.407039 25642 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.762s	user 0.000s	sys 0.003s
I20260709 12:13:57.315027 25509 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.639046 23466 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:13:57.659718 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.374081 25621 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:13:57.687218 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56649.2us
W20260709 12:13:57.643010 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 470) took 2559 ms (client timeout 2156 ms). Trace:
I20260709 12:13:57.641898 25629 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.642338 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.692891 25617 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.308315 24008 rpcz_store.cc:276] 0709 12:13:55.617757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.651264 (+ 33507us) service_pool.cc:224] Handling call
0709 12:13:55.827520 (+176256us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:56.278969 (+451449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.285080 (+  6111us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:56.594177 (+309097us) log.cc:844] Serialized 52 byte log entry
0709 12:13:56.890260 (+296083us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:56.890371 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.890507 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.123834 (+233327us) raft_consensus.cc:1685] finished
0709 12:13:57.123870 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.170527 (+ 46657us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":350286,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":15790,"thread_start_us":33148,"threads_started":1,"wal-append.queue_time_us":182999}
I20260709 12:13:57.693856 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:57.306869 25500 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.707755 23387 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:57.538838 25355 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
I20260709 12:13:57.642570 25616 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:57.681845 25621 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.706779 25629 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.635972 25148 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.694643 25358 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:57.722287 25358 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:57.723453 25358 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:57.724766 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.690337 23318 rpcz_store.cc:269] 0709 12:13:55.083489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.559329 (+475840us) service_pool.cc:224] Handling call
0709 12:13:57.642731 (+2083402us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:57.642932 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3879453056}
I20260709 12:13:57.717365 23320 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:13:57.691681 25635 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.642s	user 0.006s	sys 0.007s
I20260709 12:13:57.677896 25264 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.682314 25475 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.737747 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.685119 25408 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.739163 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.753824 25635 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.724624 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.731232 25509 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.758028 23467 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:13:57.770713 25523 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:13:57.771845 25148 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.727155 25645 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.774873 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 151215us
I20260709 12:13:57.776149 23466 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:13:57.765928 25617 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:57.787425 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.790464 25660 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:57.791620 25660 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.789415 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:57.799814 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:57.811331 23393 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:13:57.817708 25617 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:57.818367 25617 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:57.818475 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:57.818781 25264 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:57.821211 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48187.7us
I20260709 12:13:57.790710 25635 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.772683 25616 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:13:57.843750 25616 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:57.832497 25623 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.735s	user 0.009s	sys 0.005s
I20260709 12:13:57.836293 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:57.821352 25264 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:13:57.851048 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52194.6us
I20260709 12:13:57.818846 25617 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.817760 25645 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:57.794284 25509 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.861469 25616 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:13:57.861094 25264 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:57.866528 25500 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:57.880414 25674 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:13:57.861300 25635 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:13:57.883509 25408 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:57.884094 25475 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:57.884805 23318 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:13:57.885820 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:57.889596 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 67560.7us
I20260709 12:13:57.892926 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.894243 23317 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:57.894918 23321 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:13:57.882942 25468 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:57.892488 23388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.009s	sys 0.059s
I20260709 12:13:57.897682 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:57.899082 23388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.009s	sys 0.059s
I20260709 12:13:57.899278 23315 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:57.904088 23316 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:13:57.905361 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.906116 23318 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:57.892056 25629 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.907655 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.908231 23318 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:57.910938 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:57.910959 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.914601 23317 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:57.922077 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:57.922751 23395 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:57.923484 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:57.903039 25175 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:57.924351 23394 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:57.925071 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:57.931963 25509 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.938944 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:57.939194 23321 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:57.296227 23312 rpcz_store.cc:276] 0709 12:13:55.085032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.823721 (+1738689us) service_pool.cc:224] Handling call
0709 12:13:56.925650 (+101929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.926280 (+   630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.926316 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.926934 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.057595 (+130661us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:57.291446 (+233851us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.291684 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.291759 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.291799 (+    40us) raft_consensus.cc:1685] finished
0709 12:13:57.291818 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.291947 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":57504,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":1430,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":33696,"thread_start_us":323021,"threads_started":2,"wal-append.queue_time_us":153045}
W20260709 12:13:57.940858 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 86512.9us
I20260709 12:13:57.946915 23394 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:57.945932 23315 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:57.952522 23434 connection.cc:256] RPC call timeout handler was delayed by -1.55044253400024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2579s for a response
W20260709 12:13:57.966672 23434 connection.cc:256] RPC call timeout handler was delayed by -1.4190425340000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2725s for a response
I20260709 12:13:57.942859 25175 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.974885 25175 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:57.975668 23394 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:57.976294 25629 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:13:57.982060 23395 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:13:57.971361 25509 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.951668 25621 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:57.989849 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0536425340001188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3131s for a response
I20260709 12:13:57.925769 25264 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:57.991549 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:57.990228 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:57.998852 23314 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:58.000800 23394 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:13:58.003644 25509 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:57.993722 25648 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.005920 25509 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.006135 25648 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.008309 25648 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.015031 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:58.009148 25680 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:58.022056 23396 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:13:58.022253 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.022882 25680 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:58.023175 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.024304 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.024657 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.026080 23393 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:58.028999 25680 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.009366 25646 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:58.030730 25680 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.047400 25680 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.049642 25646 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.052018 25646 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:58.053995 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:58.034770 25148 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:13:58.035243 25673 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.203s	user 0.002s	sys 0.000s
I20260709 12:13:58.055436 23312 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:58.055765 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:58.055485 25148 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:58.031096 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.035668 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:58.038643 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 22fc2096e78f47158bbb4bae3a474e2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6253b834ffb744579b46fcede2fc0e91 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:13:58.039726 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.036787 23393 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:58.040714 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.056885 25148 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:58.053972 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:58.056917 25648 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:58.059772 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.060887 23390 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:58.064625 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.065321 23321 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:58.065546 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.071537 23392 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:13:58.075429 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.076182 25677 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.076802 25677 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:58.077266 25677 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.070992 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:58.086721 25678 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:13:58.087165 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.088047 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.088913 25500 raft_consensus.cc:2764] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:13:58.073693 23391 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:13:58.090021 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:58.090819 25500 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.091192 25500 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.092003 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.094373 23315 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:58.097456 25408 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.098140 25408 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.098521 25408 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.099273 25605 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:58.099901 25605 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:58.101577 25617 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.102424 23390 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:13:58.102825 25542 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:58.104002 23391 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:13:58.094578 25500 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.105886 25617 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:58.112246 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.112078 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:58.112519 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.113179 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:58.102843 23390 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 5
I20260709 12:13:58.114248 23390 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:13:58.114512 23390 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:58.118511 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.060364 25646 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:13:58.118177 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:58.119900 23287 connection.cc:256] RPC call timeout handler was delayed by -3.7899350220000088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 12:13:58.130759 23287 connection.cc:256] RPC call timeout handler was delayed by -3.7854350220000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 12:13:58.130828 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:58.121224 25605 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.132081 23315 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 12:13:58.128459 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.129035 25617 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.059911 25148 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:58.131363 23287 connection.cc:256] RPC call timeout handler was delayed by -3.6423350219999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 12:13:58.137915 23287 connection.cc:256] RPC call timeout handler was delayed by -3.6405350219999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:13:58.138417 23287 connection.cc:256] RPC call timeout handler was delayed by -3.4200350220000928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19810000000000003s for a response
W20260709 12:13:58.131132 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 95557.3us
I20260709 12:13:58.128839 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 369) took 1118 ms. Trace:
I20260709 12:13:58.139446 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:58.138796 23287 connection.cc:256] RPC call timeout handler was delayed by -3.4020350220000637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
W20260709 12:13:58.141165 23287 connection.cc:256] RPC call timeout handler was delayed by -3.3957350220000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2008s for a response
I20260709 12:13:58.145074 23980 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:13:58.145282 23287 connection.cc:256] RPC call timeout handler was delayed by -3.3606350219999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
I20260709 12:13:58.146651 25648 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:13:58.148434 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2447815010002614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 12:13:58.150048 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1112315380000837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260709 12:13:58.147625 25560 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.149849 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:58.151595 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0551103249999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 12:13:58.153506 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0437315379999745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 12:13:58.153796 23287 connection.cc:256] RPC call timeout handler was delayed by -2.866781501000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260709 12:13:58.154079 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8172815010002523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
W20260709 12:13:58.154312 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8091815010002392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
W20260709 12:13:58.154546 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8055815010002334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 12:13:58.154788 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7860103250000066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 12:13:58.155067 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7764315379999971s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
W20260709 12:13:58.155313 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7326815010001155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 12:13:58.155544 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7173815010000908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 12:13:58.155774 23287 connection.cc:256] RPC call timeout handler was delayed by -2.6219815010001639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
W20260709 12:13:58.155998 23287 connection.cc:256] RPC call timeout handler was delayed by -2.6075815010001406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 12:13:58.157819 25560 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.159982 25560 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.139236 23388 rpcz_store.cc:276] 0709 12:13:57.010700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.010962 (+   262us) service_pool.cc:224] Handling call
0709 12:13:57.011581 (+   619us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:57.106410 (+ 94829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.106460 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.107068 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.899536 (+792468us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:57.989202 (+ 89666us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.989454 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.128339 (+138885us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b54001e0fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:58.128420 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:58.128474 (+    54us) raft_consensus.cc:1685] finished
0709 12:13:58.128584 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:58.128743 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16784,"prepare.queue_time_us":70057,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":1258,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":27483,"spinlock_wait_cycles":214293376,"thread_start_us":851188,"threads_started":2,"wal-append.queue_time_us":763740}
I20260709 12:13:58.162053 25148 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.164258 23321 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:58.107189 25677 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.166369 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:58.143283 25264 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:58.150345 25635 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:13:58.190783 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45756.4us
I20260709 12:13:58.165340 25623 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:13:58.195751 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.196327 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 101563us
I20260709 12:13:58.197216 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:13:58.203195 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 52967.8us
I20260709 12:13:58.204250 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.152359 25617 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.193449 25175 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:58.193198 25403 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:13:58.201256 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.322s (SENT) [suppressed 37 similar messages]
I20260709 12:13:58.182111 25680 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:58.231050 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.233712 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.235652 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.238310 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.225385 25403 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.221328 25646 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:58.195113 25691 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:13:58.253939 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:58.254603 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50463.5us
I20260709 12:13:58.235109 25264 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:13:58.203572 25623 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.239914 25175 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:58.253636 25403 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:58.252388 25646 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:58.254906 25681 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:58.265007 25500 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.276309 25681 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:58.276794 25681 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:13:58.265982 25677 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.266633 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.280089 24015 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:13:58.266762 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.267403 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:58.281678 24014 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:58.286067 25681 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.289340 24014 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:13:58.268483 25175 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.274740 25632 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:58.275944 25680 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.268185 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:58.290380 23313 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:13:58.269237 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.259781 25264 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:58.349104 23982 connection.cc:256] RPC call timeout handler was delayed by -1.234756680999908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 12:13:58.314179 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.356629 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.355376 25500 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.362067 24016 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:13:58.363807 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:13:58.366205 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 97037.6us
I20260709 12:13:58.372211 25697 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.373031 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.373796 23461 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:58.374102 23464 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:58.375109 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.375854 24010 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:58.372324 25702 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.376812 25702 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:13:58.377025 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:58.377382 25617 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.378140 25617 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.378463 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.380093 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.380534 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.381704 25616 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:58.381976 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.382067 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.382318 25680 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:58.382778 24012 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:13:58.375111 25681 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.383675 24015 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:13:58.376255 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.378535 25617 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:58.381309 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.395617 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:58.396247 24014 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:13:58.395061 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2270361890000459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 12:13:58.377274 25702 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.398000 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.255805 25648 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.386839 25264 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:13:58.381331 24007 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:58.385196 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:58.434425 24011 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.407224 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.407224 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:58.435706 23315 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:58.372898 25697 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.398528 25617 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.412285 23465 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:13:58.412976 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.417745 25408 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.376829 25623 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.418713 24015 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:58.419456 24014 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:58.420519 25646 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.420982 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.394675 25605 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:58.423889 25264 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.425994 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:58.437178 25693 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.437582 25616 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:13:58.440641 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.441423 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.442160 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.442865 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.446645 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.448504 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:58.470609 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 373566us
W20260709 12:13:58.475030 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 93788.5us
I20260709 12:13:58.476636 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:58.437227 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:58.482317 24014 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:58.480722 23470 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:13:58.473984 23464 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:58.503856 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.486426 25623 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:13:58.497494 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:58.492655 25605 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.504520 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:58.740211 23316 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:13:58.740808 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:58.741657 25702 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.743839 25702 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.744388 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:58.744992 23318 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.745136 23316 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:13:58.742041 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:13:58.649720 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.617712 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:13:58.571135 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.747872 23315 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:58.748732 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.723572 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:58.749266 23321 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:58.593289 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 167369us
W20260709 12:13:58.602427 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.571985 24009 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:13:58.590696 25706 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:13:58.771565 23467 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:13:58.642282 24007 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:58.572609 24012 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:13:58.587110 25644 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:58.772397 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 142314us
I20260709 12:13:58.636571 25468 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.588970 25697 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.586678 25705 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.638761 25264 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.615919 25649 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:13:58.782975 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.590281 25692 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.202s
I20260709 12:13:58.572468 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:58.499348 25403 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.500862 25693 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.801203 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.801391 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.589884 25709 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.210s
I20260709 12:13:58.804930 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.806989 23394 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:13:58.796363 25661 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.011s	sys 0.019s
I20260709 12:13:58.643886 25408 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.812238 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.818118 23392 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:58.812628 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.810562 23388 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:13:58.798393 25468 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.828352 25468 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:13:58.828895 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.820938 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:13:58.829824 24008 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 1. Current term is 3. Ops: []
I20260709 12:13:58.830709 23390 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:13:58.831710 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.832355 23390 raft_consensus.cc:2368] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:13:58.821803 25403 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.821887 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.834239 23392 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4.
I20260709 12:13:58.834277 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.835976 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:58.837028 23461 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:13:58.798553 25623 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:13:58.838145 23391 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:58.824167 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:58.841566 23395 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:58.824276 25617 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.825839 25697 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:58.826435 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:13:58.817800 25661 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.011s	sys 0.019s
I20260709 12:13:58.845052 23393 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:13:58.843988 23318 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:13:58.813383 23464 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.848527 23387 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:58.842911 25468 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:13:58.844630 25697 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:13:58.808346 25705 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:58.853211 25697 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.854743 25697 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.858325 25403 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:13:58.858737 25403 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:58.859630 25403 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.860880 25403 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.863660 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.866508 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:58.867368 25623 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:58.868829 25700 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:58.870316 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:58.870993 23317 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:58.871811 23317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:58.872455 23317 raft_consensus.cc:606] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:58.872800 23317 raft_consensus.cc:614] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:58.874725 23312 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:13:58.875221 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.876000 23312 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:13:58.797798 25644 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.881884 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 375) took 1868 ms. Trace:
I20260709 12:13:58.881994 25408 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:58.882282 23396 rpcz_store.cc:276] 0709 12:13:57.013252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.013593 (+   341us) service_pool.cc:224] Handling call
0709 12:13:57.943271 (+929678us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fe9c 00007ff0a654fa5b 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:13:57.943754 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.022582 (+ 78828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.881641 (+859059us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5400121954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:58.881822 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238837760}
I20260709 12:13:58.883176 25644 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:58.883889 23313 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:13:58.884704 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.885824 25264 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.886286 25264 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:13:58.886757 25264 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:58.888543 25617 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.889671 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.892637 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:58.849206 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.850483 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:13:58.903033 23315 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:58.904227 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:13:58.911160 23318 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:13:58.912173 23321 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:58.929823 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.929989 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:13:58.930804 23312 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
W20260709 12:13:58.939268 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 94314.9us
I20260709 12:13:58.863669 25705 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:58.945971 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:58.947367 23434 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.935s (SENT)
I20260709 12:13:58.911290 25636 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:13:58.952044 25175 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:58.952574 25175 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.952191 25704 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:58.952955 25175 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:58.953301 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:58.954032 25704 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:13:58.954313 25632 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:13:58.954753 25632 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.955106 25632 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:13:58.961684 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.963107 25704 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:13:58.963663 25704 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:58.964653 25704 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.966527 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:13:58.932163 25148 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:13:58.969547 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:58.980501 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:58.983258 23318 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:13:58.976959 25175 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.985417 25175 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:58.927100 25644 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:13:58.994022 23468 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:58.949877 25636 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:58.990458 23464 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:13:59.000530 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:59.002545 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:58.943996 25701 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:13:58.971840 25709 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:13:59.008437 23318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:13:59.010288 23318 raft_consensus.cc:606] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:59.010659 23318 raft_consensus.cc:614] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:58.967387 25632 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:58.994196 25408 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:13:58.993101 25468 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.989712 25148 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:13:59.016113 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49676.9us
W20260709 12:13:59.017952 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70622.1us
I20260709 12:13:59.001264 23464 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
W20260709 12:13:59.023864 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.020690 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 60888us
I20260709 12:13:59.007566 25636 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:13:59.016187 25649 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:13:58.995817 25700 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:13:59.075786 25709 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:59.078397 25709 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:59.018445 25605 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:59.018733 25408 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:59.080415 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 143597us
W20260709 12:13:59.086794 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 192583us
I20260709 12:13:59.081625 23318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:54652
W20260709 12:13:59.082691 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.089731 23318 raft_consensus.cc:606] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:59.090122 23318 raft_consensus.cc:614] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:59.095310 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:59.096365 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:59.100579 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:59.098378 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:59.088059 25408 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:13:59.106247 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.055954 25705 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:13:59.064502 25175 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:13:59.095188 25617 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.080189 25468 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.131603 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:13:59.132146 24015 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:13:59.132265 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:59.132594 24015 pending_rounds.cc:85] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 4
I20260709 12:13:59.134600 24015 pending_rounds.cc:107] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:13:59.134917 24015 raft_consensus.cc:2889] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:59.133095 25715 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.139780 25715 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.144464 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:13:59.147006 25734 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:59.147524 25734 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:59.147812 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47298.1us
I20260709 12:13:59.148551 25735 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:13:59.148962 25735 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:59.147929 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.151104 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:59.158429 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:13:59.108577 25679 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:13:59.171576 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:59.172178 24015 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.149302 25735 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.147890 25734 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.173533 23463 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:13:59.170058 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 71751.2us
I20260709 12:13:59.185904 23980 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:13:59.133556 25617 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.199124 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:59.204747 23466 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:13:59.201014 23318 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:13:59.205595 23318 pending_rounds.cc:85] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:13:59.205941 23318 pending_rounds.cc:107] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:13:59.206233 23318 raft_consensus.cc:2889] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:59.177861 25661 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:59.227973 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:59.228312 25468 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:59.229702 25725 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.231132 25737 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:13:59.245131 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:13:59.133652 25148 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:59.250334 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:59.224947 25175 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:13:59.226771 25705 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.257766 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:59.228008 25632 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:59.264271 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 485) took 1648 ms. Trace:
I20260709 12:13:59.266353 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 483) took 2141 ms. Trace:
I20260709 12:13:59.266860 23319 rpcz_store.cc:276] 0709 12:13:57.124823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.127861 (+  3038us) service_pool.cc:224] Handling call
0709 12:13:59.266144 (+2138283us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400342954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:13:59.266279 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566071296}
I20260709 12:13:59.264806 23314 rpcz_store.cc:276] 0709 12:13:57.615809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.615962 (+   153us) service_pool.cc:224] Handling call
0709 12:13:57.643808 (+ 27846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:57.999303 (+355495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.263983 (+1264680us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400389854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:59.264193 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":816080640}
I20260709 12:13:59.271075 23395 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:13:59.274070 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:13:59.289536 23467 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:13:59.259786 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:13:59.261246 25715 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:59.293635 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63015.4us
W20260709 12:13:59.293774 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 137823us
I20260709 12:13:59.238874 25718 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:13:59.296497 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:13:59.304798 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.305866 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120114us
I20260709 12:13:59.314493 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:13:59.330632 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.299629 25727 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:13:59.293525 25175 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:59.269112 25705 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:13:59.352191 25733 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:13:59.352497 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:59.354085 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:59.355266 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:59.352197 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:13:59.357543 25358 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:13:59.364311 25729 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:13:59.394430 25738 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.394645 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 675) took 1016 ms. Trace:
I20260709 12:13:59.395210 24010 rpcz_store.cc:276] 0709 12:13:58.377839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.401690 (+ 23851us) service_pool.cc:224] Handling call
0709 12:13:59.351984 (+950294us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:13:59.394577 (+ 42593us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14397824}
I20260709 12:13:59.395262 25738 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.396991 25705 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:13:59.399921 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 62312.1us
I20260709 12:13:59.378507 25617 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:13:59.398768 23462 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:13:59.444938 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 107846us
I20260709 12:13:59.463953 23462 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:13:59.448478 25629 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.468811 25629 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:59.470892 25629 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:59.478219 25745 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:13:59.479424 25744 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:13:59.479907 25744 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:59.480924 25744 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.482625 25744 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.486444 25636 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.497291 24016 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:13:59.487293 25743 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.180s
I20260709 12:13:59.489794 25751 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:59.559578 23434 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.506s (SENT)
I20260709 12:13:59.490165 25678 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:13:59.551329 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.556648 23982 leader_election.cc:336] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.575s (SENT)
I20260709 12:13:59.446233 24007 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:13:59.561718 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 310670us
I20260709 12:13:59.562177 25661 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:59.588481 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:59.589546 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:13:59.448562 25741 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.600100 25745 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:13:59.586012 25678 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:13:59.559541 25751 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.601423 25649 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:13:59.612231 25678 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:13:59.613811 25636 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:13:59.609917 25408 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:59.621109 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 100907us
W20260709 12:13:59.633157 23434 leader_election.cc:336] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.529s (SENT)
I20260709 12:13:59.633585 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:13:59.561280 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:13:59.638878 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77686.4us
I20260709 12:13:59.645102 25408 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:13:59.619709 25661 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:13:59.596225 25175 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:13:59.714880 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.804533 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.649206 25358 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:13:59.677556 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.966883 24015 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.783s	user 0.001s	sys 0.010s
W20260709 12:13:59.871052 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 231294us
I20260709 12:14:00.311733 24015 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.312209 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:00.312448 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 690) took 1141 ms. Trace:
I20260709 12:13:59.641516 25738 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:00.306231 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 673063us
W20260709 12:14:00.314131 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.138669 23462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.002s	sys 0.011s
I20260709 12:13:59.650276 25737 consensus_queue.cc:1243] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:13:59.731339 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1650) took 1287 ms. Trace:
W20260709 12:14:00.307022 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 841671us
I20260709 12:13:59.631302 25751 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:00.315040 23469 rpcz_store.cc:276] 0709 12:13:58.438439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.438598 (+   159us) service_pool.cc:224] Handling call
0709 12:13:58.438901 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:58.785297 (+346396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.785345 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:58.785404 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:58.785465 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.725986 (+940521us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54001068d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:59.726035 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.726177 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770235648}
W20260709 12:13:59.814575 25734 consensus_meta.cc:287] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.000s
I20260709 12:14:00.313005 24015 rpcz_store.cc:276] 0709 12:13:59.170985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.171321 (+   336us) service_pool.cc:224] Handling call
0709 12:14:00.312375 (+1141054us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:59.814772 25725 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.004s
I20260709 12:14:00.307495 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1716) took 1127 ms. Trace:
W20260709 12:13:59.827955 25403 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.967s	user 0.009s	sys 0.004s
W20260709 12:13:59.681222 25694 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.534s	user 0.011s	sys 0.000s
W20260709 12:13:59.679087 25264 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.000s
W20260709 12:14:00.314745 23462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.004s	sys 0.011s
I20260709 12:13:59.682942 25678 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:13:59.661852 25649 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:13:59.674455 25759 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:14:00.304973 25735 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.110s	user 0.009s	sys 0.004s
W20260709 12:14:00.305907 25148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.018s
W20260709 12:14:00.284695 24012 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.976s	user 0.009s	sys 0.001s
W20260709 12:14:00.358098 25148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.009s	sys 0.018s
I20260709 12:14:00.358225 24012 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.324549 24013 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.359688 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 692) took 1148 ms. Trace:
I20260709 12:14:00.360065 24012 rpcz_store.cc:276] 0709 12:13:59.211859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.215928 (+  4069us) service_pool.cc:224] Handling call
0709 12:14:00.359622 (+1143694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:00.360769 25735 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.361512 25747 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.362015 25747 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.363225 25735 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:00.329630 25264 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:00.320415 23315 rpcz_store.cc:276] 0709 12:13:59.179654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.179777 (+   123us) service_pool.cc:224] Handling call
0709 12:13:59.180285 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.180680 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:13:59.364425 (+183745us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.454213 (+ 89788us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.597795 (+143582us) log.cc:844] Serialized 572 byte log entry
0709 12:13:59.597967 (+   172us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:13:59.597999 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.598071 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.307225 (+709154us) raft_consensus.cc:1685] finished
0709 12:14:00.307245 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.307430 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126895,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":62,"raft.queue_time_us":11060,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":89329,"thread_start_us":222524,"threads_started":2,"wal-append.queue_time_us":390984}
I20260709 12:14:00.369715 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 414) took 1424 ms. Trace:
I20260709 12:14:00.370203 23391 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:00.370162 23467 rpcz_store.cc:276] 0709 12:13:58.945491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.980148 (+ 34657us) service_pool.cc:224] Handling call
0709 12:14:00.369526 (+1389378us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54001063d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:00.369642 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708406144}
I20260709 12:14:00.372131 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1401) took 1122 ms. Trace:
I20260709 12:14:00.372479 23391 rpcz_store.cc:276] 0709 12:13:59.249712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.249881 (+   169us) service_pool.cc:224] Handling call
0709 12:13:59.262082 (+ 12201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.370707 (+1108625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.371926 (+  1219us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000a8554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:00.372080 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548083072}
I20260709 12:13:59.654774 25727 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:13:59.648805 25741 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.323972 25725 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:00.322638 25734 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.344521 25621 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.345963 25523 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:00.346832 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1669) took 1243 ms. Trace:
I20260709 12:14:00.347143 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 474) took 5262 ms. Trace:
I20260709 12:14:00.384722 24013 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.312433 25358 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.349671 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.385118 23320 rpcz_store.cc:276] 0709 12:13:55.084747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.799017 (+1714270us) service_pool.cc:224] Handling call
0709 12:13:57.675861 (+876844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:57.725874 (+ 50013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.346876 (+2621002us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54003c0350. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:00.347062 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942643072}
I20260709 12:13:59.673553 25739 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.751s
I20260709 12:14:00.388064 25741 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.384958 23462 rpcz_store.cc:276] 0709 12:13:59.103733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.588649 (+484916us) service_pool.cc:224] Handling call
0709 12:13:59.589116 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.601521 (+ 12405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.601553 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.602712 (+  1159us) log.cc:844] Serialized 572 byte log entry
0709 12:14:00.323584 (+720872us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:00.323627 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.323707 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.346599 (+ 22892us) raft_consensus.cc:1685] finished
0709 12:14:00.346622 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.346764 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":720397,"threads_started":1,"wal-append.queue_time_us":720979}
I20260709 12:13:59.973232 25760 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.391486 23463 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:00.391914 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.392685 25739 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:00.399971 23463 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:00.394189 25660 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:00.401077 23463 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.402185 23463 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.402374 25660 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.402576 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.403467 25739 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:00.392477 25714 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:00.403952 25739 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:00.404187 25714 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:00.404701 25714 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.403946 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:00.395812 25264 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.389809 25727 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.400581 25760 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:00.398887 25358 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:00.408461 25760 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.404992 25739 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.402828 25660 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.414324 25660 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.414803 25714 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.394734 25734 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.413637 25760 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:00.416361 25760 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:00.391974 25747 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.419778 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:00.420627 25760 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:00.421922 25760 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:00.422845 25760 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.421298 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.426182 23391 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
W20260709 12:14:00.431216 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 1.93184e+06us
I20260709 12:14:00.432854 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.433805 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:00.435302 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:00.435904 24015 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:00.436863 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:00.437395 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:00.438755 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:00.439337 23462 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.439670 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:00.437521 24007 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.440200 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.440342 24010 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:00.440953 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.441139 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.441630 24010 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:14:00.442155 23392 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:00.442318 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.442667 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:00.442936 23388 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:00.443126 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.443794 23387 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:14:00.444162 24012 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:00.443267 24015 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:00.441676 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.446393 23462 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:00.446854 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.447614 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.447817 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:00.451833 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1720) took 1200 ms. Trace:
I20260709 12:14:00.452358 23318 rpcz_store.cc:276] 0709 12:13:59.251219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.255322 (+  4103us) service_pool.cc:224] Handling call
0709 12:13:59.274833 (+ 19511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.292945 (+ 18112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.292995 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.303707 (+ 10712us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.371825 (+ 68118us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:13:59.371876 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.371936 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.451548 (+1079612us) raft_consensus.cc:1685] finished
0709 12:14:00.451581 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.451753 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5307,"thread_start_us":23411,"threads_started":1,"wal-append.queue_time_us":121600}
I20260709 12:14:00.456082 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.457038 25621 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:00.457702 25621 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:00.465354 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.471704 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.501667 25358 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.533399 25358 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.534912 25646 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.473695 25775 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:14:00.519393 25723 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.414s	user 0.016s	sys 0.003s
W20260709 12:14:00.521181 25728 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 1.252s	user 0.002s	sys 0.013s
I20260709 12:14:00.523247 25635 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:00.474220 23463 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:00.473732 25768 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:00.475544 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:00.570204 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 114195us
I20260709 12:14:00.567137 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 416) took 1596 ms. Trace:
W20260709 12:14:00.530606 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 325) took 13022 ms (client timeout 15000 ms). Trace:
I20260709 12:14:00.571806 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.569049 25635 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.570850 23463 rpcz_store.cc:276] 0709 12:13:58.970425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.329977 (+359552us) service_pool.cc:224] Handling call
0709 12:13:59.348844 (+ 18867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.391383 (+1042539us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400213520. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:14:00.566758 (+175375us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.567062 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188096256}
W20260709 12:14:00.570214 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 94758.1us
I20260709 12:14:00.560786 23462 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:00.577544 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.575726 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:00.582015 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.577188 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:00.583314 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:00.573055 25697 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 12:14:00.585057 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.584015 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:14:00.594388 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.582744 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:00.474524 25714 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.597580 23389 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:00.574227 25775 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:14:00.601100 23434 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.552s (SENT)
I20260709 12:14:00.601923 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:00.584472 25700 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.594628 25768 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:14:00.623170 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51445.2us
I20260709 12:14:00.602160 25739 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.624783 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.621516 25175 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.627028 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.604592 25775 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.599301 25779 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:00.628268 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.605846 25718 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:00.597820 25264 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:14:00.571441 23217 rpcz_store.cc:269] 0709 12:13:47.508248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.519012 (+ 10764us) service_pool.cc:224] Handling call
0709 12:14:00.530392 (+13011380us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:56.632051 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:56.672907 (+ 40856us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:56.672969 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:13:56.672995 (+    26us) tablet.cc:662] Decoding operations
0709 12:13:56.675306 (+  2311us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:56.675385 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:56.675425 (+    40us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:13:56.677115 (+  1690us) tablet.cc:701] Row locks acquired
0709 12:13:56.677137 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:13:56.677329 (+   192us) write_op.cc:270] Start()
0709 12:13:56.677485 (+   156us) write_op.cc:276] Timestamp: P: 1783599236677268 usec, L: 0
0709 12:13:56.677510 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:13:56.684105 (+  6595us) log.cc:844] Serialized 5183 byte log entry
0709 12:13:57.536867 (+852762us) op_driver.cc:464] REPLICATION: finished
0709 12:13:57.692646 (+155779us) write_op.cc:301] APPLY: starting
0709 12:13:57.692817 (+   171us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:57.693316 (+   499us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:57.693668 (+   352us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:57.729203 (+ 35535us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:57.809781 (+ 80578us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:13:57.809863 (+    82us) write_op.cc:312] APPLY: finished
0709 12:13:57.810455 (+   592us) log.cc:844] Serialized 95 byte log entry
0709 12:13:57.810675 (+   220us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:57.810918 (+   243us) write_op.cc:454] Released schema lock
0709 12:13:57.810995 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:57.811621 (+   626us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:57.811656 (+    35us) write_op.cc:454] Released schema lock
0709 12:14:00.200129 (+2388473us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a967132a 00007ff0a95f6f14 00007ff0a95f6ec3 00007ff0a95f6e38 00007ff0a95f6df3 00007ff0a95f60dc 00007ff0a961fe16 00007ff0a9607cbb 00007ff0a95f239d 00007ff0a95f5d75
Metrics: {"mutex_wait_us":214800,"reported_tablets":25,"spinlock_wait_cycles":2944,"child_traces":[["op",{"apply.queue_time_us":136083,"apply.run_cpu_time_us":10164,"apply.run_wall_time_us":123330,"num_ops":13,"prepare.queue_time_us":992773,"prepare.run_cpu_time_us":31625,"prepare.run_wall_time_us":398557,"raft.queue_time_us":304619,"raft.run_cpu_time_us":35682,"raft.run_wall_time_us":2798283,"replication_time_us":859063,"spinlock_wait_cycles":125058432,"thread_start_us":826795,"threads_started":4,"wal-append.queue_time_us":301012,"wal-append.run_cpu_time_us":33808,"wal-append.run_wall_time_us":1826871}]]}
I20260709 12:14:00.636755 25693 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.629616 25775 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.639496 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 493) took 2382 ms. Trace:
I20260709 12:14:00.639933 24008 rpcz_store.cc:276] 0709 12:13:58.256987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.258480 (+  1493us) service_pool.cc:224] Handling call
0709 12:13:58.350549 (+ 92069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:58.830213 (+479664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.639188 (+1808975us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54007c01d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:00.639406 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056398592}
I20260709 12:14:00.641693 23464 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:00.643980 24011 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:00.645464 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 693) took 1411 ms. Trace:
I20260709 12:14:00.645747 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:00.645859 24013 rpcz_store.cc:276] 0709 12:13:59.234231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.234366 (+   135us) service_pool.cc:224] Handling call
0709 12:14:00.645278 (+1410912us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54006bdad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:00.645382 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2460330240}
I20260709 12:14:00.646778 24007 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.649462 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:00.622115 25768 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:00.654197 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53131.8us
W20260709 12:14:00.655169 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0137179290002223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1811s for a response
I20260709 12:14:00.626585 25739 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.617065 25358 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:14:00.664067 25714 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:00.666038 24013 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:14:00.758575 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.662381 25768 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.631670 25175 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.648808 25737 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.715950 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.632673 25629 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:00.663328 25738 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:00.668114 25739 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.774704 25794 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:00.775126 25738 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:00.792340 25738 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.793527 25738 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.715790 25660 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.795588 25660 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:00.782009 25785 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:14:00.797497 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.797912 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:00.798342 25779 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:00.798815 25629 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:00.733774 25714 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:00.623548 25700 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.773988 25747 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.797937 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1397) took 1549 ms. Trace:
W20260709 12:14:00.802234 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 175290us
I20260709 12:14:00.816382 23389 rpcz_store.cc:276] 0709 12:13:59.248365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.257370 (+  9005us) service_pool.cc:224] Handling call
0709 12:13:59.278678 (+ 21308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.599104 (+1320426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.797710 (+198606us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002dfd54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:00.797874 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3032649472}
I20260709 12:14:00.799000 25779 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.784765 25751 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.821671 25779 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.821784 25714 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.822733 25358 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.823330 25751 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.823226 25358 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:00.802598 25621 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:00.824831 25621 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.826915 25621 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.827092 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:00.828055 25358 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.829493 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:00.830356 25784 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.831081 23467 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:00.830803 25784 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:00.831521 23436 leader_election.cc:336] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.527s (SENT)
I20260709 12:14:00.831960 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:00.817663 25660 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:14:00.832996 25784 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:00.833066 25751 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.834424 25784 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.834326 25747 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.834863 25784 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.835237 25784 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.799388 25629 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:00.838577 25791 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:00.839617 25791 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:00.839867 24007 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:00.839980 25791 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:00.841239 25751 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:00.841022 25791 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.842289 25791 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.822305 25779 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:00.837220 25629 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.846698 25175 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.841396 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:00.849844 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:00.864244 25542 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:00.865883 25542 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.863529 25175 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:00.866287 25542 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.803017 25737 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:00.874255 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 206211us
I20260709 12:14:00.875387 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1653) took 2436 ms. Trace:
W20260709 12:14:00.881096 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.882593 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 217016us
W20260709 12:14:00.883287 23434 leader_election.cc:336] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.689s (SENT)
W20260709 12:14:00.886344 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.887284 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:00.888312 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:00.892032 25721 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.796s	user 0.012s	sys 0.000s
W20260709 12:14:00.893496 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 399) took 3367 ms (client timeout 1542 ms). Trace:
I20260709 12:14:00.870556 25635 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:00.893882 23468 rpcz_store.cc:269] 0709 12:13:57.560465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.564578 (+  4113us) service_pool.cc:224] Handling call
0709 12:14:00.893275 (+3328697us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002256d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:00.893381 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3162902912}
I20260709 12:14:00.895013 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1706) took 2151 ms. Trace:
I20260709 12:14:00.895422 25635 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.895356 23316 rpcz_store.cc:276] 0709 12:13:58.743125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.793849 (+ 50724us) service_pool.cc:224] Handling call
0709 12:13:58.807928 (+ 14079us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.819750 (+ 11822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.819797 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.843285 (+ 23488us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.951052 (+107767us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:58.951092 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.951159 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.891470 (+1940311us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:00.894741 (+  3271us) raft_consensus.cc:1685] finished
0709 12:14:00.894759 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.894954 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10906,"spinlock_wait_cycles":57228032,"thread_start_us":95265,"threads_started":1,"wal-append.queue_time_us":164748}
I20260709 12:14:00.897082 25737 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.902136 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:00.869501 25700 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:00.906893 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.865902 24008 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:00.866611 25175 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:00.888047 23467 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.879945 23465 rpcz_store.cc:276] 0709 12:13:58.439394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.864778 (+425384us) service_pool.cc:224] Handling call
0709 12:13:58.865153 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.894916 (+ 29763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.894944 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.913229 (+ 18285us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.055514 (+142285us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:13:59.055578 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.055677 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.870546 (+1814869us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b240010c088. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a640d9c4 00007ff0a640bacb 00007ff0a65c6dc9 00007ff0a65c6a55 00007ff0a65c5fb0 00007ff0a923c488 00007ff0a923cbe8 00007ff0a9214398 00007ff0a920f76a 00007ff0a920ebb6
0709 12:14:00.875068 (+  4522us) raft_consensus.cc:1685] finished
0709 12:14:00.875087 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.875324 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46122,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":16025,"spinlock_wait_cycles":91768064,"thread_start_us":86116,"threads_started":1,"wal-append.queue_time_us":196707}
I20260709 12:14:00.897815 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:00.910892 23312 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.914261 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:00.899705 25661 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.915114 24015 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:00.919560 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:00.922647 23314 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:00.921006 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:00.923987 23321 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:00.930049 23315 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:00.888814 25629 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.931308 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:00.933218 25737 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:00.942166 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:00.873545 25775 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.945571 25775 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:00.946470 25629 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.947083 25768 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.933938 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:00.949833 23320 raft_consensus.cc:606] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:00.950292 23320 raft_consensus.cc:614] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:00.951854 25542 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.952561 25175 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:00.934508 25621 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:00.955359 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:00.956126 24007 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:00.908993 25779 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.934813 25661 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:00.977440 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 45346.2us
W20260709 12:14:00.991101 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102854us
I20260709 12:14:00.967739 23312 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
W20260709 12:14:01.117885 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 234610us
I20260709 12:14:00.988514 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52264 (request call id 89) took 1737 ms. Trace:
I20260709 12:14:01.117902 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:00.914325 25794 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:01.121505 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.744s (SENT)
W20260709 12:14:01.121277 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 220705us
I20260709 12:14:01.123797 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.135392 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.128190 23321 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:01.129565 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 620) took 2707 ms. Trace:
I20260709 12:14:01.122838 23394 rpcz_store.cc:276] 0709 12:13:59.250790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.257081 (+  6291us) service_pool.cc:224] Handling call
0709 12:14:00.988100 (+1731019us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002e9854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a967a735 00007ff0a96729fa 00007ff0ae63a4c9 00007ff0a7bb6c54 00007ff0a7bb6be2 00007ff0a7bb6b11 00007ff0a7bb6a8c 00007ff0a7bb5d42 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:00.988368 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085687040}
I20260709 12:14:00.985250 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 625) took 2561 ms. Trace:
I20260709 12:14:01.140394 23466 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:00.949299 25768 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:01.139331 24014 rpcz_store.cc:276] 0709 12:13:58.422138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.474851 (+ 52713us) service_pool.cc:224] Handling call
0709 12:13:58.482141 (+  7290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:58.483554 (+  1413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.129336 (+2645782us) spinlock_profiling.cc:243] Waited 6.99 ms on lock 0x7b540011d5d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:01.129504 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16075008}
I20260709 12:14:01.142213 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:00.983171 25746 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.706s	user 0.008s	sys 0.004s
I20260709 12:14:01.139683 24011 rpcz_store.cc:276] 0709 12:13:58.423438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.562112 (+138674us) service_pool.cc:224] Handling call
0709 12:14:00.643763 (+2081651us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:00.644394 (+   631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.984915 (+340521us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54007c01d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:00.985150 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4729597184}
I20260709 12:14:01.072551 25644 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:14:01.150863 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.168347 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:01.171201 23466 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.116434 25629 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:01.130532 25700 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.182432 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.181397 24015 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:01.189009 23389 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:00.914584 25735 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.226s
I20260709 12:14:01.191962 23395 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:01.193742 23391 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:01.166462 25776 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:01.196040 23396 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:01.140609 25702 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.135078 25743 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:14:01.145004 25697 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:14:01.034977 25739 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.252562 25739 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:01.255016 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 22fc2096e78f47158bbb4bae3a474e2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:14:01.255823 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.256909 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.205744 23465 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:01.258625 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.129869 25737 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:14:01.194458 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.197207 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:01.260048 23394 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.260205 24007 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.139122 25784 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.262228 25784 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:01.263145 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 93246.5us
I20260709 12:14:01.263942 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.267652 25629 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.268155 25629 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:01.268569 25629 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.273627 25800 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.199610 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.265174 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.285939 23466 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:01.130785 25779 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.241734 25542 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:01.193293 25648 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.243124 25661 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:01.243526 25610 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:01.210906 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.340034 23387 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.211234 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.173323 25644 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:01.337116 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 215654us
I20260709 12:14:01.197705 25794 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:01.210981 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:01.358060 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 234347us
I20260709 12:14:01.290397 25621 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:01.360914 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 186508us
W20260709 12:14:01.368573 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 103496us
I20260709 12:14:01.376358 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.356354 25644 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:01.371239 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.811s (SENT)
I20260709 12:14:01.379449 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.335223 25702 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:01.367285 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 407) took 2455 ms. Trace:
I20260709 12:14:01.394790 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:01.395411 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.333930 25776 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.355394 25610 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.406559 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:01.407346 23467 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:01.408690 23390 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:14:01.418285 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.401088 25621 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:01.422794 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 156786us
I20260709 12:14:01.425725 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:01.426534 23466 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.432946 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:01.438618 23982 connection.cc:256] RPC call timeout handler was delayed by -1.0765826609999749s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:14:01.441115 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:01.441955 23468 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:01.448009 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:01.448784 24015 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 7: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:01.470724 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:01.393345 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 243782us
I20260709 12:14:01.472510 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.473301 23396 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:14:01.454608 24011 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:01.479418 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.482788 24015 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:01.438843 25805 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.491122 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.492609 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:01.446424 25681 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:01.446662 25775 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:01.500584 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.518162 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.518978 23392 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.477485 25751 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.480026 25697 consensus_queue.cc:1243] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:14:01.501711 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 125444us
I20260709 12:14:01.505755 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.445730 25794 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:01.507081 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:01.509502 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 138246us
I20260709 12:14:01.527745 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:01.528084 23395 raft_consensus.cc:2368] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:01.510915 23466 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:01.512166 24014 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:01.514043 23391 raft_consensus.cc:2368] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:14:01.515566 24010 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:14:01.532420 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108490us
I20260709 12:14:01.517236 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.519927 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.520212 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.523066 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.533633 23396 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:01.530923 23468 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:01.517135 24015 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:01.446612 25743 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:14:01.507344 25681 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.537360 23389 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.534404 23468 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.538811 23388 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.549630 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.553519 25812 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:01.553967 25812 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.554659 23392 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:01.554833 25812 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.556335 25812 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:01.559157 25809 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:01.559754 25809 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.561632 24007 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:01.563160 23468 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.564316 23468 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:01.567373 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1672) took 1135 ms. Trace:
I20260709 12:14:01.567812 23464 rpcz_store.cc:276] 0709 12:14:00.432003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.437123 (+  5120us) service_pool.cc:224] Handling call
0709 12:14:01.567183 (+1130060us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400230fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:01.567305 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467804032}
I20260709 12:14:01.571394 25779 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:01.391726 23461 rpcz_store.cc:276] 0709 12:13:58.905776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.908464 (+  2688us) service_pool.cc:224] Handling call
0709 12:13:58.971650 (+ 63186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.016876 (+ 45226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.018698 (+  1822us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.099182 (+ 80484us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.286897 (+187715us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:13:59.291383 (+  4486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.297667 (+  6284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.982408 (+1684741us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:01.290017 (+307609us) raft_consensus.cc:1685] finished
0709 12:14:01.290038 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:01.358917 (+ 68879us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":53363,"spinlock_wait_cycles":6004992,"thread_start_us":112649,"threads_started":1,"wal-append.queue_time_us":150944}
I20260709 12:14:01.574345 25648 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:01.527431 25751 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.574795 25648 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.575680 25648 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:01.578416 25776 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:01.541069 25175 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:14:01.524787 25810 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:01.582594 25701 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:01.583034 25701 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:01.584173 25800 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.531400 25264 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:14:01.588380 25747 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:01.589504 25702 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:01.590940 25806 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:14:01.595023 25629 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:01.596356 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.524634 25700 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:14:01.597973 23318 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:01.523097 25737 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 12:14:01.599066 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.599678 23312 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:01.599843 23316 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:01.600646 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.393445 25660 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Committing config change with OpId 5.7: config changed from index 5 to 7, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:01.601258 23312 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:01.601812 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.602494 23316 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:01.603253 23312 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:01.603668 23317 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:01.606317 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:01.606916 23319 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:01.614240 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71580.5us
W20260709 12:14:01.614936 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 81654.4us
I20260709 12:14:01.583922 25701 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.618836 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.576879 25648 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.628137 23313 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:14:01.544668 23434 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.889s (SENT)
I20260709 12:14:01.625377 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 687) took 2512 ms. Trace:
I20260709 12:14:01.526042 25775 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:01.629608 24016 rpcz_store.cc:276] 0709 12:13:59.113561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.113759 (+   198us) service_pool.cc:224] Handling call
0709 12:13:59.114134 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:59.497704 (+383570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.624963 (+2127259us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54006301d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:01.625280 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":880661248}
I20260709 12:14:01.630569 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:01.635952 23320 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:01.635988 23318 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.637214 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.630429 25791 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:01.539700 25794 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.640393 25692 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:14:01.642565 25791 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:01.631189 25811 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:14:01.576897 25751 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:01.643064 25791 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.610558 25681 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:01.644014 24008 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.001s
I20260709 12:14:01.644395 24008 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:01.644614 25644 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.645735 25775 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.560994 25809 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:01.651647 25737 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:01.652074 25737 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.652875 25737 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.654286 25737 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.656634 25791 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.658160 25794 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:01.661850 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 130155us
I20260709 12:14:01.662312 25610 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:01.662519 23434 leader_election.cc:336] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.975s (SENT)
I20260709 12:14:01.666759 23318 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:01.667964 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:01.672509 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 130992us
I20260709 12:14:01.673561 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.674427 23321 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:01.675030 23321 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.675361 23321 raft_consensus.cc:614] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:01.677021 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:01.677659 23436 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 2.246s (SENT)
W20260709 12:14:01.680262 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61560.3us
W20260709 12:14:01.681538 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.629650 25800 raft_consensus.cc:2764] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:14:01.685699 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:01.687376 24016 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:01.687861 23388 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.686560 23390 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:01.688611 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:01.689375 24015 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:01.695196 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.991s (SENT)
I20260709 12:14:01.641680 25701 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.522199 25805 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.661788 25629 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:01.697665 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:01.698395 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:01.692399 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:01.710305 24014 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:01.701784 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:01.701846 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:01.703205 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 153645us
I20260709 12:14:01.703714 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.715723 23468 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:14:01.716207 23468 pending_rounds.cc:85] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 4
I20260709 12:14:01.716594 23468 pending_rounds.cc:107] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:14:01.717312 24015 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:01.717183 23468 raft_consensus.cc:2889] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.694870 25743 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:01.718632 25743 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.719039 23312 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:01.719277 25737 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:01.719759 25737 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.719465 25743 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.720127 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.720860 25743 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:01.720630 25737 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.721406 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:01.722334 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.678881 25175 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.711024 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:01.726071 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.726641 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:01.729113 25779 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.729952 25800 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:01.730384 25800 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:01.730655 25644 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:01.730744 25800 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:01.695714 25633 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:01.733563 25633 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:01.734637 25791 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:01.734450 25633 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.735180 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:01.684597 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.739032 24013 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:01.723207 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:01.744366 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59900.8us
I20260709 12:14:01.745245 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.751883 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.721877 25737 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.723749 24011 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:14:01.757372 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 278400us
W20260709 12:14:01.765842 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70671.2us
I20260709 12:14:01.766279 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:01.766912 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.047s (SENT)
I20260709 12:14:01.698412 25692 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:01.771657 23319 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:01.773138 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:01.774998 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.695216 25720 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 12:14:01.803785 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.065s (SENT)
I20260709 12:14:01.750895 25629 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:01.778669 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:01.774932 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:01.795629 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.811350 23319 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:01.768888 25175 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:01.777247 24009 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:01.777935 23314 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:01.777117 25810 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.801736 25649 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:01.826253 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.776918 25698 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:01.832209 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52619.3us
I20260709 12:14:01.775880 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.768187 25811 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 12:14:01.845728 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 69947.8us
I20260709 12:14:01.823043 25734 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:01.800799 25798 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:14:01.866811 24013 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:01.867431 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 88843.1us
I20260709 12:14:01.867985 23319 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:01.868350 23319 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:01.861925 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.843072 25741 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 12:14:01.848402 23434 leader_election.cc:336] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.073s (SENT)
I20260709 12:14:01.853971 25812 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:01.870154 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.880384 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:01.881314 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.881340 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:01.884912 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.885103 23466 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:01.882833 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:01.886802 23466 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:01.870836 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.872763 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:01.892198 23468 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:01.896870 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58803.9us
W20260709 12:14:01.899093 23434 leader_election.cc:336] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.131s (SENT)
I20260709 12:14:01.900328 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:01.860993 25734 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.823848 25714 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 12:14:01.901643 25775 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:01.845705 25175 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:01.826980 25815 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:01.901260 23219 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:01.855326 25649 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:01.867658 25500 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:01.836378 25264 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:14:01.809267 25692 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:01.931756 23434 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.141s (SENT)
I20260709 12:14:01.934280 23319 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:01.906183 25751 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:01.914374 25734 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.942332 23219 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 1 to 2, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:01.954789 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:01.930529 23314 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:01.950410 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:01.918594 25781 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:01.923144 23466 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:01.967173 25629 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:01.968102 25629 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:01.968321 23319 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:01.967207 25649 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.969453 23219 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:01.973908 23434 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.149s (SENT)
I20260709 12:14:01.974421 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:01.963900 23466 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 5 to 7, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:01.968667 25629 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:01.943701 25817 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:01.906010 25776 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:14:01.939718 25743 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:01.953809 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:01.989281 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:01.961319 25781 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:01.995222 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:01.975163 25720 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:01.995275 23319 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:14:02.022192 23319 pending_rounds.cc:85] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 3
I20260709 12:14:02.022528 23319 pending_rounds.cc:107] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:02.022800 23319 raft_consensus.cc:2889] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.995954 23314 raft_consensus.cc:2376] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
I20260709 12:14:01.971508 25500 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:02.019950 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:01.985244 25812 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:02.023165 23219 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:02.026382 23318 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:01.966010 25751 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:01.994166 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:02.036475 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62601.2us
I20260709 12:14:02.000633 25779 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:02.041880 23434 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.201s (SENT)
I20260709 12:14:02.035952 23219 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:02.018730 25776 consensus_queue.cc:1243] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:02.051280 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:02.023403 25805 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:02.019134 25743 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:02.090857 25830 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:14:02.092450 25829 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 12:14:02.073086 25827 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:14:02.043614 25779 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:02.206420 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.214072 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.230943 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.047539 25175 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:02.156665 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.242945 25779 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:02.250309 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.255529 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 213692us
I20260709 12:14:02.282399 23318 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:14:02.282722 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.203s (SENT)
W20260709 12:14:02.256289 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.261521 23319 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 5 to 7, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:02.283320 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:02.289688 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 295576us
I20260709 12:14:02.291240 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:02.254765 25828 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.278s	user 0.009s	sys 0.003s
W20260709 12:14:02.305366 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 425922us
W20260709 12:14:02.298472 23434 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.322s (SENT)
I20260709 12:14:02.290351 25500 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:02.282667 25823 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.305s	user 0.010s	sys 0.000s
I20260709 12:14:02.274675 25742 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.248497 25812 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:02.359387 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.360708 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.361697 23318 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:02.273058 25810 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:02.358944 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:02.319870 25776 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:02.428797 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 69931.7us
W20260709 12:14:02.337776 25835 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.269s	user 0.000s	sys 0.026s
I20260709 12:14:02.343297 25817 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.433060 25817 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.438272 25817 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:02.367457 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58606us
I20260709 12:14:02.423769 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.440384 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.441061 23314 raft_consensus.cc:2376] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4. Re-sending same reply.
W20260709 12:14:02.441681 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0709040189999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 12:14:02.442018 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.445314 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:02.461516 23434 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.167s (SENT)
I20260709 12:14:02.462105 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:02.461539 23314 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:02.446291 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.450220 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.378430 25833 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:14:02.383191 25800 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.652s	user 0.017s	sys 0.002s
W20260709 12:14:02.386026 25822 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:14:02.380390 25834 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:14:02.384954 25825 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.342s	user 0.010s	sys 0.000s
I20260709 12:14:02.353062 25760 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:14:02.389863 25812 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.449173 25701 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.000s
I20260709 12:14:02.487605 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.496486 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 205325us
I20260709 12:14:02.497326 23316 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:14:02.324620 25781 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.472182 25775 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:02.412492 25358 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 12:14:02.475117 25744 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:02.393679 25660 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 12:14:02.424036 25738 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:14:02.478811 25817 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:02.390520 25845 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:14:02.383951 25742 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:02.519162 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 57714.9us
W20260709 12:14:02.520105 23434 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.579s (SENT)
I20260709 12:14:02.501012 25692 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:02.525640 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:02.525903 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.488168 25800 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.527875 23318 raft_consensus.cc:2410] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:14:02.515506 25744 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:02.538496 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 147143us
I20260709 12:14:02.524312 25697 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:02.521339 25743 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.531941 25781 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:02.561676 23434 leader_election.cc:336] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.725s (SENT)
I20260709 12:14:02.562305 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:02.497105 25812 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.499377 25847 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:14:02.553381 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.570968 23318 raft_consensus.cc:2368] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6.
I20260709 12:14:02.531813 25817 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:02.520889 25775 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:02.552838 25734 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.003s
W20260709 12:14:02.547322 25648 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.917s	user 0.008s	sys 0.003s
W20260709 12:14:02.574539 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.579636 23314 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:14:02.578236 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.583879 23315 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:02.586035 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 420) took 3199 ms. Trace:
W20260709 12:14:02.589339 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 117942us
I20260709 12:14:02.595212 25775 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.603478 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.604075 25836 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.358s	user 0.000s	sys 0.002s
I20260709 12:14:02.604350 23318 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:02.605732 25851 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:14:02.606576 25841 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:14:02.606667 25800 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.577816 25840 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.607795 25840 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.609617 25678 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:02.610056 25678 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:02.610338 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:02.611016 24012 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:02.610960 25678 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.579860 25734 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.615365 25784 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:02.615887 25734 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:02.577603 25743 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:02.586697 23469 rpcz_store.cc:276] 0709 12:13:59.386435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.321082 (+934647us) service_pool.cc:224] Handling call
0709 12:14:00.330928 (+  9846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:00.331284 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:00.584639 (+253355us) log.cc:844] Serialized 14 byte log entry
0709 12:14:01.689175 (+1104536us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b18002386c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:14:01.834080 (+144905us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:01.868282 (+ 34202us) log.cc:844] Serialized 26 byte log entry
0709 12:14:01.868460 (+   178us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:01.868493 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:01.868555 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:02.578773 (+710218us) raft_consensus.cc:1685] finished
0709 12:14:02.585723 (+  6950us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:02.585955 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17853,"prepare.queue_time_us":286595,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1504,"raft.queue_time_us":53853,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":14065,"spinlock_wait_cycles":10000768,"thread_start_us":397320,"threads_started":2,"wal-append.queue_time_us":1350290}
I20260709 12:14:02.624365 23314 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:02.612162 25678 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:02.616534 25737 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.843s	user 0.009s	sys 0.000s
W20260709 12:14:02.620324 25681 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.977s	user 0.012s	sys 0.000s
I20260709 12:14:02.586524 25817 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:02.591655 25649 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.630586 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.637972 23318 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:02.631656 23316 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:02.610476 25840 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.626124 25800 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:02.640661 25800 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.644645 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.664671 23434 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.246s (SENT)
I20260709 12:14:02.665295 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:02.684762 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.670739 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 240887us
I20260709 12:14:02.695541 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:02.622511 25744 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.696990 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.698940 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:02.700280 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 151062us
I20260709 12:14:02.701038 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:02.710434 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:02.714458 23318 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:02.716540 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:02.638288 25681 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.714479 23396 raft_consensus.cc:2368] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
W20260709 12:14:02.661963 25815 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.000s
W20260709 12:14:02.720695 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.724102 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:02.730504 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65879.8us
W20260709 12:14:02.731482 23434 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.386s (SENT)
W20260709 12:14:02.699417 25629 consensus_meta.cc:287] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.709s	user 0.006s	sys 0.006s
I20260709 12:14:02.711859 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.640864 25649 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.702366 25840 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.736330 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:02.742121 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:02.640236 25817 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:02.747494 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.750304 23319 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:02.725234 25742 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:02.716668 25859 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:02.745226 25629 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:02.756137 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.769781 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.784490 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.790545 23319 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:02.744263 25815 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:02.753707 23434 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.131s (SENT)
I20260709 12:14:02.875707 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:02.876804 25854 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:02.842499 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118094us
I20260709 12:14:02.771251 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.731153 25852 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:02.891002 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.835717 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.874773 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.909871 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.911314 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.770941 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.754221 23393 raft_consensus.cc:2368] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:02.776743 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.924878 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:02.878062 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 124395us
I20260709 12:14:02.740520 25863 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:02.883281 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 153230us
W20260709 12:14:02.778123 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.931391 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.752552 25849 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:02.769006 25702 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:02.764849 25813 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:02.764302 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 712) took 1632 ms. Trace:
I20260709 12:14:02.934648 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:02.947654 23469 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:02.771680 25809 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:02.942384 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.201s (SENT)
I20260709 12:14:02.886391 25859 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:02.930781 25743 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.940498 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.986704 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.940544 23318 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:02.938728 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:02.765568 25848 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:02.762737 25649 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.944304 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:02.981201 25812 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.935936 25863 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:02.984229 25860 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:14:02.948282 25784 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.928960 25500 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.772401 25760 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.184s
I20260709 12:14:02.776288 25840 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.015087 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.173470 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1696) took 1294 ms. Trace:
W20260709 12:14:03.179353 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 248854us
W20260709 12:14:03.100072 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 167734us
W20260709 12:14:03.180102 23434 leader_election.cc:336] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.202s (SENT)
W20260709 12:14:03.099740 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 170379us
I20260709 12:14:03.183166 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.184190 23318 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:03.183092 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.185074 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.179508 23463 rpcz_store.cc:276] 0709 12:14:01.879078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.881419 (+  2341us) service_pool.cc:224] Handling call
0709 12:14:01.881709 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.974031 (+ 92322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:01.974064 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:01.974121 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:01.974151 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.173183 (+1199032us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:03.173234 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.173378 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211850624}
I20260709 12:14:03.185911 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:03.163872 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.195480 23314 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:03.198367 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.227s (SENT)
I20260709 12:14:03.198843 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.199070 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.200168 23314 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 4.
W20260709 12:14:03.200934 25678 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.003s
I20260709 12:14:03.201143 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.202425 23314 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 4.
I20260709 12:14:03.203440 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.204749 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:02.946964 25702 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:02.955740 25813 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:03.015640 25858 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:03.186619 23318 raft_consensus.cc:2368] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
I20260709 12:14:03.177198 25768 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:14:03.017750 25853 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:14:03.216140 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:03.172237 25660 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:03.220171 23389 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:03.175031 25741 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.019212 25629 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.227408 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.238921 23318 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:03.239995 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.016705 25681 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.016505 25817 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:02.956100 24009 rpcz_store.cc:276] 0709 12:14:01.131885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.134354 (+  2469us) service_pool.cc:224] Handling call
0709 12:14:02.764075 (+1629721us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54001df454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:02.764236 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757052800}
I20260709 12:14:03.241282 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.246728 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.247463 23318 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.192601 25863 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:03.199079 25500 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.257968 25500 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.244298 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.219211 25812 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.259999 25618 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.253726 25864 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:03.260603 25618 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.260928 25864 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.261544 25859 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.265637 23312 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:03.185845 25782 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:14:03.267266 23393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 403) took 1782 ms (client timeout 1818 ms). Trace:
W20260709 12:14:03.267655 23393 rpcz_store.cc:269] 0709 12:14:01.485204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.485424 (+   220us) service_pool.cc:224] Handling call
0709 12:14:03.267055 (+1781631us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400190854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:03.267185 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821320448}
I20260709 12:14:03.254627 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.268625 23318 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.215075 25870 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:03.254315 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55990.7us
W20260709 12:14:03.300828 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.304s (SENT)
I20260709 12:14:03.301230 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.269577 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.270709 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.302650 23319 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:03.303021 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.297865 25864 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.132522 25701 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 12:14:03.230587 25768 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.310217 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.310226 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.021826 25743 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:03.322256 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.229861 25660 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:03.322053 23318 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.244952 25649 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.323339 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.329056 23319 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.311584 25884 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.328588 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.332573 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.208525 25784 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.333467 23319 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:03.335121 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 153928us
W20260709 12:14:03.338294 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.339012 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.721s (SENT)
I20260709 12:14:03.339478 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.342555 23318 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:03.343722 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.334349 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.346892 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:03.326261 25779 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.002s	user 0.011s	sys 0.001s
I20260709 12:14:03.347481 23319 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.350827 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.351138 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:03.310953 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94859.8us
I20260709 12:14:03.327759 25812 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.325968 25768 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.354576 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.247165 25741 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.355283 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1415) took 1687 ms. Trace:
I20260709 12:14:03.336565 25800 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.359395 23392 rpcz_store.cc:276] 0709 12:14:01.667628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.667749 (+   121us) service_pool.cc:224] Handling call
0709 12:14:01.668185 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:01.679394 (+ 11209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:01.679424 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:01.679900 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.423571 (+743671us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b18003ef380. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9
0709 12:14:02.572245 (+148674us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:02.572285 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.572364 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.352477 (+780113us) raft_consensus.cc:1685] finished
0709 12:14:03.352506 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.355198 (+  2692us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":13313,"spinlock_wait_cycles":13481088,"thread_start_us":148454,"threads_started":1,"wal-append.queue_time_us":1035022}
I20260709 12:14:03.358379 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:03.358841 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 420493us
I20260709 12:14:03.363752 23318 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6.
I20260709 12:14:03.367918 23320 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.375918 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.368386 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.378087 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.379720 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.355755 25737 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.355957 25856 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.383447 23360 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.385327 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.389608 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.390372 23320 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.385622 23319 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:03.389393 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.391615 23318 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:03.330137 25702 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.362741 25883 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.396880 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.355225 25779 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:03.382436 25781 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.001s
I20260709 12:14:03.326084 25870 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.364737 25618 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.417887 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:03.438521 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.439101 23320 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:14:03.439914 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46885.3us
I20260709 12:14:03.441318 23319 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.442446 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:03.440102 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56762.1us
I20260709 12:14:03.445398 23318 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.446664 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.447800 23318 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6.
I20260709 12:14:03.447872 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.378497 25500 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:14:03.366652 25886 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:14:03.454226 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.455466 23317 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:03.360773 25660 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.340993 25813 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:03.449756 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80831.8us
I20260709 12:14:03.448577 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.458597 23319 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:03.448555 23320 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:03.445389 25883 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:03.460346 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.450093 25744 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:03.464504 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.438325 25800 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.448828 25874 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.392380 25737 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.390821 25884 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:14:03.474217 23318 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:03.475732 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.477042 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.462492 25817 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.464771 25702 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.464948 25781 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:03.472097 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.465112 25840 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:03.536111 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.544253 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:03.536816 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72385.4us
I20260709 12:14:03.504621 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.538439 23320 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:03.547506 23318 raft_consensus.cc:2376] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4. Re-sending same reply.
I20260709 12:14:03.473776 25874 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:03.552655 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.553472 23320 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:03.538985 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.479611 24012 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.000s
I20260709 12:14:03.461797 25863 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.554533 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:03.558236 25878 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:14:03.558676 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.559435 23320 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.559237 25876 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.174s
W20260709 12:14:03.559968 23461 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.536s	user 0.014s	sys 0.000s
I20260709 12:14:03.560320 23461 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
W20260709 12:14:03.560827 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 436) took 1688 ms (client timeout 2066 ms). Trace:
I20260709 12:14:03.560920 25889 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.143s
I20260709 12:14:03.558076 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:03.560174 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.562283 23320 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:03.477265 25800 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.570043 25897 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:03.571569 25751 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:03.561182 23461 rpcz_store.cc:269] 0709 12:14:01.872673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.875959 (+  3286us) service_pool.cc:224] Handling call
0709 12:14:03.560771 (+1684812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:03.574326 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:03.551875 25875 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:14:03.538546 25864 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.579372 25882 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:03.581218 23392 raft_consensus.cc:1217] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Deduplicated request from leader. Original: 1.4->[2.5-3.6]   Dedup: 3.6->[]
I20260709 12:14:03.539836 25784 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.567763 25697 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.563290 25856 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.562969 25720 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.591094 23392 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: [2.5-3.6]
I20260709 12:14:03.593271 23390 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:03.563903 25874 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.565124 24012 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:03.596232 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 129466us
I20260709 12:14:03.569229 23982 leader_election.cc:400] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:03.542676 25817 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:03.609211 25871 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
W20260709 12:14:03.608677 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55734.3us
I20260709 12:14:03.612082 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:03.609419 25879 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:14:03.607125 25720 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.573403 25870 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.594311 25882 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:03.579025 25883 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.596127 25743 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.618556 23982 leader_election.cc:400] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
W20260709 12:14:03.622400 25880 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 12:14:03.623026 25890 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:14:03.623965 25894 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:14:03.625705 25882 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:03.633136 23982 leader_election.cc:403] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:03.609627 25892 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.596692 25864 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.617120 25751 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:03.618629 24012 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 704) took 3006 ms (client timeout 2142 ms). Trace:
I20260709 12:14:03.632881 25744 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:03.636740 25649 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.608639 25856 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.649540 25883 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5.
I20260709 12:14:03.653048 25800 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:03.654068 25809 consensus_meta.cc:287] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.000s
I20260709 12:14:03.654768 23462 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:03.655148 25852 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.655577 25852 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.642899 25779 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.642347 25892 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.663944 25892 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:03.647101 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:03.664865 25882 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.667613 25882 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.668124 25892 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:03.670446 25892 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:03.672247 25892 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.650746 25649 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.675889 25649 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.678592 25856 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.645725 25751 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.681906 25751 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:03.644867 24012 rpcz_store.cc:269] 0709 12:14:00.605371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.605905 (+   534us) service_pool.cc:224] Handling call
0709 12:14:01.682749 (+1076844us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:02.610217 (+927468us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:03.611852 (+1001635us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56666240}
I20260709 12:14:03.669009 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.669157 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.688215 23315 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.688902 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.690881 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.691311 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.691504 25720 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:03.691747 23315 raft_consensus.cc:2410] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:03.691953 24010 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:03.692276 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.692526 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.692876 24016 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.693068 23315 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.693787 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.694355 24013 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.693801 24011 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:03.696748 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.696913 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.698303 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 518) took 1281 ms. Trace:
I20260709 12:14:03.698850 23313 rpcz_store.cc:276] 0709 12:14:02.416450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.470041 (+ 53591us) service_pool.cc:224] Handling call
0709 12:14:02.470337 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.696699 (+226362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.696730 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.712261 (+ 15531us) log.cc:844] Serialized 26 byte log entry
0709 12:14:02.967773 (+255512us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:02.967835 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.967941 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.698014 (+730073us) raft_consensus.cc:1685] finished
0709 12:14:03.698045 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.698223 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156578,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":1994,"thread_start_us":422855,"threads_started":2,"wal-append.queue_time_us":611259}
I20260709 12:14:03.700119 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.701164 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.701805 24012 raft_consensus.cc:2376] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6. Re-sending same reply.
I20260709 12:14:03.700510 23313 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:03.703840 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.704537 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.695116 25805 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:03.696024 25737 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:03.702975 25744 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.697489 24009 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:03.709420 25744 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.712270 25744 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.713694 25883 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.714210 25883 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:03.723574 23467 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1698) took 1844 ms (client timeout 1960 ms). Trace:
I20260709 12:14:03.723727 23466 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:14:03.724076 23467 rpcz_store.cc:269] 0709 12:14:01.879604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.880169 (+   565us) service_pool.cc:224] Handling call
0709 12:14:03.723355 (+1843186us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54003edad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:03.723507 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1593206528}
I20260709 12:14:03.726178 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.727103 23315 raft_consensus.cc:2376] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4. Re-sending same reply.
I20260709 12:14:03.698305 25633 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:03.756606 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.689158 25768 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:03.788362 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.803952 25779 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.804330 25849 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.804901 25849 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.804608 25779 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:03.805688 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.806710 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.815455 23315 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:03.819257 25779 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.822140 25870 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.833683 25870 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.836081 25870 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:03.836737 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.837872 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.838146 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:03.838436 23316 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:03.838713 24014 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.838816 25863 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:03.839269 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.839859 23316 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:03.840621 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.841192 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.841625 23467 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:03.822089 25768 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:03.843755 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.844444 23318 raft_consensus.cc:2368] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6.
I20260709 12:14:03.846071 25848 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:03.846516 25848 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:03.848798 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.823591 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52264 (request call id 90) took 1482 ms. Trace:
W20260709 12:14:03.849316 23362 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 2.142s (SENT) [suppressed 37 similar messages]
I20260709 12:14:03.849586 23388 rpcz_store.cc:276] 0709 12:14:02.341653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.341825 (+   172us) service_pool.cc:224] Handling call
0709 12:14:03.823280 (+1481455us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b5400090720. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95f1520 00007ff0a967ab42 00007ff0a9679800 00007ff0a96729fa 00007ff0ae63a4c9 00007ff0a7bb6c54 00007ff0a7bb6be2 00007ff0a7bb6b11 00007ff0a7bb6a8c 00007ff0a7bb5d42
0709 12:14:03.823530 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309801344}
I20260709 12:14:03.845129 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.823001 25817 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:03.851717 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:03.858608 23467 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.852833 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.853219 23320 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:03.858852 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:03.859324 23318 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:03.854718 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 513) took 1453 ms. Trace:
I20260709 12:14:03.861256 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.862008 24014 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:03.862955 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:03.863442 24014 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:03.832055 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:03.857193 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.857314 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.865375 23316 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.865455 23313 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.866425 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:03.866638 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:03.867129 23461 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:03.867403 23464 raft_consensus.cc:2410] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:14:03.868551 23470 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:03.868983 25882 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.869374 25882 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.869733 25882 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.869894 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:03.871999 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.873106 23982 leader_election.cc:400] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:03.873332 23982 leader_election.cc:403] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:14:03.874686 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.875164 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:03.875748 23316 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:03.875909 23313 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:03.860118 23321 rpcz_store.cc:276] 0709 12:14:02.401417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.413098 (+ 11681us) service_pool.cc:224] Handling call
0709 12:14:02.413478 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.413693 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:02.476942 (+ 63249us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.710283 (+1233341us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b18002f6f20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:14:03.852757 (+142474us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:03.852805 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:03.852859 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.852942 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.854660 (+  1718us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30141,"prepare.queue_time_us":63067,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":3384,"raft.queue_time_us":19851,"raft.run_wall_time_us":5614,"spinlock_wait_cycles":4326272,"thread_start_us":105504,"threads_started":1}
I20260709 12:14:03.875860 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:03.877113 25775 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:03.879603 25897 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:03.879616 25775 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:03.877130 25738 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:03.875033 25863 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:03.880458 25738 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.880920 25813 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:03.880792 25775 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:03.877321 25782 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:14:03.880923 25738 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:03.881186 25739 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.878584 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:03.961383 23396 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:03.888659 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:03.878595 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:03.922539 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.017472 23313 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:03.970265 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:04.018289 23318 raft_consensus.cc:2376] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4. Re-sending same reply.
I20260709 12:14:04.014256 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:04.013608 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 137815us
I20260709 12:14:04.019985 23982 leader_election.cc:400] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:04.020327 23982 leader_election.cc:403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:14:03.925850 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:03.957530 23434 leader_election.cc:336] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.516s (SENT)
I20260709 12:14:03.994683 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:04.015537 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 137007us
I20260709 12:14:03.922570 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.021147 24014 raft_consensus.cc:2410] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:14:03.924999 23362 leader_election.cc:336] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Timed out: RequestConsensusVote RPC to 127.21.150.195:33909 timed out after 1.960s (SENT)
W20260709 12:14:04.022289 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 97338.9us
I20260709 12:14:04.020967 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.023554 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.023324 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:04.024430 23463 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:04.023520 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:04.023967 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66523.6us
I20260709 12:14:04.028515 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:04.025332 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.029249 23321 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:04.025556 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:04.030786 23321 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:14:04.031450 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1305) took 1260 ms. Trace:
I20260709 12:14:04.031782 23321 rpcz_store.cc:276] 0709 12:14:02.770718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.030264 (+1259546us) service_pool.cc:224] Handling call
0709 12:14:04.030574 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.031200 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.031394 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.032904 23321 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:04.034039 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1307) took 1261 ms. Trace:
I20260709 12:14:04.034595 23321 rpcz_store.cc:276] 0709 12:14:02.771921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.032474 (+1260553us) service_pool.cc:224] Handling call
0709 12:14:04.032724 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.033257 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.033418 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.036098 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.036552 23390 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.037484 23467 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:04.038257 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.038802 23467 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:04.039743 23467 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:14:04.040280 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:03.875380 25870 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:14:04.041783 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.042654 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:03.916543 25744 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:14:04.044726 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.034718 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:04.050472 25714 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:04.050866 25714 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.051227 25714 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.053084 25681 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:04.054762 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.055575 23464 raft_consensus.cc:2410] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:04.057405 25681 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:04.064986 23313 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:04.065037 25911 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.066388 23313 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:04.066325 25911 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.067942 23313 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:04.069244 23313 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:03.993479 25864 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:04.046028 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:04.073529 25714 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.879137 25892 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:04.074366 25812 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.074530 23318 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:04.074923 25812 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.076084 25714 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:04.076773 23287 connection.cc:256] RPC call timeout handler was delayed by -4.5663704509997842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 12:14:04.046075 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.068769 25911 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:04.077343 23318 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
W20260709 12:14:04.077184 23287 connection.cc:256] RPC call timeout handler was delayed by -4.5663704509997842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
I20260709 12:14:04.077708 23470 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:04.077980 23287 connection.cc:256] RPC call timeout handler was delayed by -4.5015704509999068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 12:14:04.078292 23287 connection.cc:256] RPC call timeout handler was delayed by -4.497970450999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 12:14:04.078563 23287 connection.cc:256] RPC call timeout handler was delayed by -4.4934704509998937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 12:14:04.078848 23287 connection.cc:256] RPC call timeout handler was delayed by -4.3818704509997133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 12:14:04.079113 23287 connection.cc:256] RPC call timeout handler was delayed by -4.3818704509997133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 12:14:04.079483 23287 connection.cc:256] RPC call timeout handler was delayed by -4.344970450999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
W20260709 12:14:04.079761 23287 connection.cc:256] RPC call timeout handler was delayed by -4.12357045099975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 12:14:04.080026 23287 connection.cc:256] RPC call timeout handler was delayed by -4.0866704509999181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 12:14:04.080286 23287 connection.cc:256] RPC call timeout handler was delayed by -4.0866704509999181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 12:14:04.080542 23287 connection.cc:256] RPC call timeout handler was delayed by -4.0605704509998759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:14:04.080814 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2532704509999348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3049s for a response
W20260709 12:14:04.081067 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9769704509997155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3356s for a response
I20260709 12:14:04.077068 25812 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.073508 25912 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.082412 25912 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.084491 25912 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.085553 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:04.086539 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.086874 25864 raft_consensus.cc:2764] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:03.958086 25779 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.088004 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.087649 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:03.931430 25883 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:04.024117 25175 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.089584 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:04.089299 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46701.7us
W20260709 12:14:04.090140 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.021940 25782 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:04.047542 25633 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:04.092283 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.033627 25817 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:04.089486 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.034016 25768 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.041175 25819 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:04.104384 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:04.086539 23470 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:04.119917 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.091589 25912 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:04.010922 25739 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:03.994839 25874 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.110881 23391 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:04.111416 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.048657 25744 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:04.077791 25911 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.117666 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:04.150771 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.137703 25907 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.127s	user 0.000s	sys 0.005s
I20260709 12:14:04.151782 25840 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:04.125830 25864 raft_consensus.cc:2764] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:04.108191 25884 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:14:04.120474 23470 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.108178 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.101739 25888 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:04.109373 23463 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:04.122921 25782 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:04.121170 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:04.178980 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 57881.1us
I20260709 12:14:04.131466 25697 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.131687 25739 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.131057 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:04.095958 25896 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:04.015909 25849 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.161163 23469 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:14:04.197295 23470 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:04.198539 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.199211 23470 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:04.165643 25811 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.165542 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:04.165936 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 515) took 1752 ms. Trace:
I20260709 12:14:04.170383 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1306) took 1398 ms. Trace:
I20260709 12:14:04.172501 23391 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:04.174961 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.143160 25819 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.109678 25175 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.157974 25738 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:04.184064 25905 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:04.185768 25741 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.186197 25848 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.189664 25899 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:04.190289 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:04.191184 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 22fc2096e78f47158bbb4bae3a474e2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:14:04.239850 23206 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48709.8us
I20260709 12:14:04.119387 25779 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.191929 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:04.192227 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.240903 23314 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:04.193610 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.121320 25768 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.190378 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:04.156901 25915 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:04.117045 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74490.2us
W20260709 12:14:04.219895 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 74685.3us
I20260709 12:14:04.252624 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.220503 23312 rpcz_store.cc:276] 0709 12:14:02.771573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.955504 (+183931us) service_pool.cc:224] Handling call
0709 12:14:04.170166 (+1214662us) spinlock_profiling.cc:243] Waited 86.1 ms on lock 0x7b54000ec754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:04.170310 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197926272}
I20260709 12:14:04.224092 23317 rpcz_store.cc:276] 0709 12:14:02.413667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.413880 (+   213us) service_pool.cc:224] Handling call
0709 12:14:02.470976 (+ 57096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.456944 (+985968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.165268 (+708324us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b5400275bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:04.165463 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263716736}
I20260709 12:14:04.156193 25681 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:14:04.260063 23392 raft_consensus.cc:2368] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:04.172966 25888 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.226725 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.191951 25697 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.148332 25913 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:04.275635 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.233764 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.151926 25911 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.236393 23470 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:04.276432 23463 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:04.195698 25896 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.194789 25863 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:04.245746 24014 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:04.257190 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.259014 23395 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
W20260709 12:14:04.261874 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68328.1us
I20260709 12:14:04.242678 25899 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.274695 23314 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:14:04.262727 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 113597us
I20260709 12:14:04.233654 25912 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:04.277364 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.278927 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:04.286432 23470 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.286526 23312 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:04.280951 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.291349 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:04.291690 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:04.162346 25744 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.285399 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.225757 25811 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.292676 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:04.292725 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.265852 25888 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.266134 25817 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.297204 23470 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:04.298695 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:04.339574 23317 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:04.300365 25738 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:04.344584 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59260.3us
I20260709 12:14:04.303330 25848 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:04.304282 25837 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 2.022s	user 0.005s	sys 0.002s
I20260709 12:14:04.305815 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:04.307298 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.306806 23390 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:04.307371 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.363693 23463 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:04.365003 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.365847 23463 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:04.366976 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.368319 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.310043 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.310655 25779 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.312011 25697 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.314117 25874 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:04.370604 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.371392 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:04.371592 25697 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.372463 25697 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:04.317543 25864 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:04.292747 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.321645 25819 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.321027 24012 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Committing config change with OpId 5.7: config changed from index 5 to 7, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:04.324054 25849 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.324241 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1292) took 1558 ms. Trace:
I20260709 12:14:04.286118 25912 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.285683 25782 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.390944 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.263864 25915 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.329913 25899 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.334642 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.265100 25810 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.277720 25813 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:04.339589 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.375150 25681 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:04.281279 25896 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:04.347476 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 90377.9us
I20260709 12:14:04.347607 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:04.292598 25913 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:04.314568 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.363229 23388 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8. Re-sending same reply.
I20260709 12:14:04.390136 25912 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.379448 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:04.440815 23463 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:04.441190 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:04.441800 24014 raft_consensus.cc:2376] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
I20260709 12:14:04.380797 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:04.381762 23320 rpcz_store.cc:276] 0709 12:14:02.765948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.562824 (+796876us) service_pool.cc:224] Handling call
0709 12:14:03.853052 (+290228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.859285 (+  6233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.323962 (+464677us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540001de50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:04.324157 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666841344}
W20260709 12:14:04.382913 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 90625.9us
I20260709 12:14:04.442440 23317 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.339769 25812 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:04.389492 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.328169 25768 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:04.395849 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61300.8us
I20260709 12:14:04.347764 25817 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.452282 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.452841 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:04.411791 23312 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:04.338331 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.391081 25738 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:04.329108 25882 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.438012 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.442747 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.442811 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:04.482488 23320 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:04.447049 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57664.3us
I20260709 12:14:04.390609 25895 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:14:04.453758 24014 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:04.418581 25697 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:04.417865 25813 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:04.493937 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 202302us
I20260709 12:14:04.495064 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:04.480248 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:04.485829 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63719.6us
I20260709 12:14:04.487686 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.498834 23388 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:04.490001 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47334.2us
W20260709 12:14:04.508504 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.527493 23320 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:04.528901 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:04.558183 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.593252 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:04.446775 25812 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.612042 23315 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 12:14:04.612741 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83917.5us
I20260709 12:14:04.452543 25681 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.617856 23390 raft_consensus.cc:2368] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
W20260709 12:14:04.621623 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 123214us
I20260709 12:14:04.625362 23320 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:14:04.624748 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81337.6us
W20260709 12:14:04.626497 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 172776us
I20260709 12:14:04.417630 25810 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:04.488519 25697 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.495066 25848 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:04.494321 25738 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.589537 25661 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:04.488345 25863 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:04.589387 25813 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:04.616212 25913 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.621165 25812 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:04.650841 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 155847us
I20260709 12:14:04.634222 25739 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:04.643940 25697 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:04.682646 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:04.617823 25707 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:14:04.690145 23320 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:14:04.693976 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:04.695618 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.647660 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:04.701139 23388 raft_consensus.cc:2368] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:04.709538 25738 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:04.710398 25919 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:14:04.652365 25930 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:04.711760 25930 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.712267 25930 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:04.712925 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1768) took 1162 ms. Trace:
I20260709 12:14:04.713337 23316 rpcz_store.cc:276] 0709 12:14:03.550255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.133177 (+582922us) service_pool.cc:224] Handling call
0709 12:14:04.142667 (+  9490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.148481 (+  5814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.148525 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.156363 (+  7838us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.283668 (+127305us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:04.283711 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.283769 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.712722 (+428953us) raft_consensus.cc:1685] finished
0709 12:14:04.712742 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.712866 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7483,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":105794,"threads_started":1,"wal-append.queue_time_us":252904}
I20260709 12:14:04.692056 25924 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:14:04.692288 25911 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.624748 25681 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.700696 25849 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:04.730549 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.733773 25810 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.734356 25810 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:04.734112 25935 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:04.690740 25913 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:04.736119 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.747677 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.748706 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:04.717473 25811 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:04.763202 25849 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:04.733639 25915 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:04.766336 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:04.745667 25911 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:04.777983 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.783329 25739 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.755530 25779 raft_consensus.cc:2764] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:14:04.771847 25811 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:04.793308 23287 leader_election.cc:400] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:04.802248 25661 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:04.818032 23287 leader_election.cc:403] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:14:04.775926 25849 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:04.835678 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:04.846746 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 345828us
W20260709 12:14:04.817279 25918 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 12:14:04.786872 25934 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.884408 25849 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:04.888718 25849 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:04.902966 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 220761us
I20260709 12:14:04.904101 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.905375 23317 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:04.906507 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:04.817582 25882 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:04.908648 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 279742us
I20260709 12:14:04.912810 23434 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:04.764705 25863 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:04.939257 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.886193 25779 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:04.908718 25924 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:04.958652 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.963594 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:04.934206 25935 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:04.967058 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89486.7us
I20260709 12:14:04.965202 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:04.948284 25831 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 2.450s	user 0.004s	sys 0.000s
W20260709 12:14:05.044639 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 128954us
I20260709 12:14:05.048259 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.054659 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:05.055335 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.056960 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.062553 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1761) took 1608 ms. Trace:
I20260709 12:14:05.063108 23313 rpcz_store.cc:276] 0709 12:14:03.454507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.070013 (+615506us) service_pool.cc:224] Handling call
0709 12:14:04.070282 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.149456 (+ 79174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.149496 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.149960 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.243797 (+ 93837us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:04.243841 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.243905 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.062230 (+818325us) raft_consensus.cc:1685] finished
0709 12:14:05.062264 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.062470 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3707,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":172089,"threads_started":2,"wal-append.queue_time_us":188385}
I20260709 12:14:05.065415 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:05.066561 23313 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:14:05.067930 25931 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 12:14:04.965195 25935 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:05.069877 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:04.905910 25934 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.093060 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:05.094411 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:05.069010 25842 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 2.570s	user 0.001s	sys 0.015s
I20260709 12:14:05.097554 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:05.099280 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:05.100139 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.101795 23469 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:05.123279 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 159753us
W20260709 12:14:05.164674 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.166971 23320 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:14:04.941890 25739 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:05.169653 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:04.961858 25779 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.056676 25947 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:05.093457 25935 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:05.200098 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2193) took 1106 ms. Trace:
I20260709 12:14:05.061412 25661 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.200708 23388 rpcz_store.cc:276] 0709 12:14:04.093624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.068633 (+975009us) service_pool.cc:224] Handling call
0709 12:14:05.200009 (+131376us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.206898 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.234586 25959 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:05.236724 25959 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:05.180814 25932 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.409s	user 0.000s	sys 0.001s
I20260709 12:14:05.243247 23313 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:05.248350 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:05.246218 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1781) took 1055 ms. Trace:
I20260709 12:14:05.197579 25947 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:05.251999 23392 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:05.249704 23313 rpcz_store.cc:276] 0709 12:14:04.191168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.245143 (+1053975us) service_pool.cc:224] Handling call
0709 12:14:05.245568 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.245801 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.245842 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.245899 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:05.245953 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.246023 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.246151 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.345901 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:05.349560 23463 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.011s	sys 0.043s
W20260709 12:14:05.352933 23463 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.011s	sys 0.043s
I20260709 12:14:05.173547 25697 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.379804 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1782) took 1188 ms. Trace:
I20260709 12:14:05.380450 23313 rpcz_store.cc:276] 0709 12:14:04.191413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.342046 (+1150633us) service_pool.cc:224] Handling call
0709 12:14:05.342467 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.342714 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.342751 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.342807 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:05.342855 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.342923 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.379702 (+ 36779us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.385051 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.252980 25913 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:05.431882 23469 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:05.443892 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.344951 25947 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:05.462218 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:05.179423 25950 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.214s
W20260709 12:14:05.398535 25926 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.564s	user 0.001s	sys 0.001s
W20260709 12:14:05.561371 25949 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:14:05.562914 25943 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.563687 25943 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.244526 25959 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.686261 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.687227 23388 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:05.713896 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 422) took 1790 ms. Trace:
I20260709 12:14:05.714342 23388 rpcz_store.cc:276] 0709 12:14:03.923289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.530054 (+1606765us) service_pool.cc:224] Handling call
0709 12:14:05.713812 (+183758us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.738513 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:05.740087 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:05.677599 25929 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.779s	user 0.000s	sys 0.002s
W20260709 12:14:05.499552 23312 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.009s	sys 0.022s
I20260709 12:14:05.504123 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1783) took 1191 ms. Trace:
I20260709 12:14:05.790575 23313 rpcz_store.cc:276] 0709 12:14:04.191661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.381696 (+1190035us) service_pool.cc:224] Handling call
0709 12:14:05.382162 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.382400 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.382441 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.382500 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:05.382543 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.382615 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.382741 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.793025 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1784) took 1598 ms. Trace:
I20260709 12:14:05.383062 25911 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:05.507706 23317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.059s	sys 0.104s
I20260709 12:14:05.658715 25913 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:05.793767 23317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.059s	sys 0.105s
I20260709 12:14:05.793392 23313 rpcz_store.cc:276] 0709 12:14:04.194558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.792095 (+1597537us) service_pool.cc:224] Handling call
0709 12:14:05.792448 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.792673 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.792698 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.792749 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:05.792786 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.792842 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.792970 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:05.795156 23436 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.037s	sys 0.073s
W20260709 12:14:05.795568 23436 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.037s	sys 0.073s
W20260709 12:14:05.795866 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 668844us
I20260709 12:14:05.798691 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1788) took 1411 ms. Trace:
I20260709 12:14:05.855414 23313 rpcz_store.cc:276] 0709 12:14:04.387722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.794644 (+1406922us) service_pool.cc:224] Handling call
0709 12:14:05.794920 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.798310 (+  3390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.798340 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.798390 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:05.798426 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.798500 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.798631 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:05.858003 23467 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.765s	user 0.025s	sys 0.023s
W20260709 12:14:05.858458 23467 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.025s	sys 0.023s
I20260709 12:14:05.863243 23313 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:14:05.864398 23313 pending_rounds.cc:85] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:14:05.864681 23313 pending_rounds.cc:107] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:14:05.865134 23313 raft_consensus.cc:2889] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:05.849924 23312 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.009s	sys 0.022s
I20260709 12:14:05.840739 23388 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:14:05.866859 23388 pending_rounds.cc:85] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 6
I20260709 12:14:05.870296 23388 pending_rounds.cc:107] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:14:05.870970 23388 raft_consensus.cc:2889] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:05.871898 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1423) took 1495 ms. Trace:
W20260709 12:14:05.872812 23316 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.107s	user 0.024s	sys 0.158s
W20260709 12:14:05.873335 23316 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.025s	sys 0.158s
W20260709 12:14:05.789234 25738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.002s	sys 0.010s
I20260709 12:14:05.873454 25943 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:05.867347 25911 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:05.861878 25913 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:05.884464 25849 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.995s	user 0.012s	sys 0.000s
I20260709 12:14:05.895777 25849 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.897042 25911 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:05.897666 25911 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:05.867324 25811 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.029s	sys 0.046s
I20260709 12:14:05.898133 25913 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:05.898581 25811 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.029s	sys 0.049s
I20260709 12:14:05.899026 25849 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:05.899150 25913 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.898195 25911 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:05.872340 23388 rpcz_store.cc:276] 0709 12:14:04.376858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.741153 (+1364295us) service_pool.cc:224] Handling call
0709 12:14:05.840223 (+ 99070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.871560 (+ 31337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.871827 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.875934 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 541) took 1483 ms. Trace:
W20260709 12:14:05.795774 25945 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.027s	sys 0.032s
I20260709 12:14:05.925800 23388 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:14:05.926860 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2203) took 1089 ms. Trace:
I20260709 12:14:05.927435 23388 rpcz_store.cc:276] 0709 12:14:04.837774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.922521 (+1084747us) service_pool.cc:224] Handling call
0709 12:14:05.925445 (+  2924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.926470 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.926779 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.905264 23313 rpcz_store.cc:276] 0709 12:14:04.392572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.856655 (+1464083us) service_pool.cc:224] Handling call
0709 12:14:05.857085 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.875512 (+ 18427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.875860 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:05.900769 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.034s	sys 0.163s
I20260709 12:14:05.932503 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 542) took 1535 ms. Trace:
W20260709 12:14:05.933751 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.035s	sys 0.163s
W20260709 12:14:05.934911 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 880348us
I20260709 12:14:05.933840 23313 rpcz_store.cc:276] 0709 12:14:04.397346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.929826 (+1532480us) service_pool.cc:224] Handling call
0709 12:14:05.930216 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.931398 (+  1182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.931479 (+    81us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.931618 (+   139us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:05.931750 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.931947 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.932310 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.940413 23388 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:05.944185 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 543) took 1545 ms. Trace:
W20260709 12:14:05.903350 25738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.002s	sys 0.014s
I20260709 12:14:05.949227 23313 rpcz_store.cc:276] 0709 12:14:04.398490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.943306 (+1544816us) service_pool.cc:224] Handling call
0709 12:14:05.943602 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.943786 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.943822 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.943870 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:05.943917 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.943985 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.944109 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.959283 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 544) took 1541 ms. Trace:
I20260709 12:14:05.960739 23313 rpcz_store.cc:276] 0709 12:14:04.417786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.958198 (+1540412us) service_pool.cc:224] Handling call
0709 12:14:05.958605 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.958849 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.958890 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.958947 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:05.959000 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.959053 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.959205 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.954816 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:05.967422 23313 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:05.971894 25958 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:14:05.972019 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1789) took 1007 ms. Trace:
I20260709 12:14:05.974924 23313 rpcz_store.cc:276] 0709 12:14:04.964460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.966824 (+1002364us) service_pool.cc:224] Handling call
0709 12:14:05.967239 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.971754 (+  4515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.971962 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.977841 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1771) took 2407 ms. Trace:
I20260709 12:14:05.980865 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1790) took 1016 ms. Trace:
I20260709 12:14:05.983783 23313 rpcz_store.cc:276] 0709 12:14:04.964788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.977859 (+1013071us) service_pool.cc:224] Handling call
0709 12:14:05.978200 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.980524 (+  2324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.980551 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:05.980595 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:05.980625 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.980682 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.980806 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:05.978396 23314 rpcz_store.cc:276] 0709 12:14:03.569951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.286468 (+716517us) service_pool.cc:224] Handling call
0709 12:14:04.343206 (+ 56738us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:04.355994 (+ 12788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.356032 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:04.366430 (+ 10398us) log.cc:844] Serialized 598 byte log entry
0709 12:14:04.600485 (+234055us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:04.600551 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.600669 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:05.977521 (+1376852us) raft_consensus.cc:1685] finished
0709 12:14:05.977551 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:05.977759 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26126,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":120488,"threads_started":1,"wal-append.queue_time_us":409501}
W20260709 12:14:05.962361 25953 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.739s	user 0.001s	sys 0.000s
W20260709 12:14:05.997550 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1420) took 2453 ms (client timeout 2622 ms). Trace:
W20260709 12:14:06.017840 23396 rpcz_store.cc:269] 0709 12:14:03.544476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.544587 (+   111us) service_pool.cc:224] Handling call
0709 12:14:05.997244 (+2452657us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:05.997397 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933991808}
I20260709 12:14:06.010422 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.020587 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 647) took 1513 ms (client timeout 1615 ms). Trace:
W20260709 12:14:06.021076 24011 rpcz_store.cc:269] 0709 12:14:04.507311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.010033 (+1502722us) service_pool.cc:224] Handling call
0709 12:14:06.020518 (+ 10485us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.012844 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58143.5us
I20260709 12:14:05.989405 25697 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.024343 25660 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:06.010892 25956 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:06.025002 25956 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.025346 25956 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:05.975806 25888 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.628s	user 0.011s	sys 0.000s
W20260709 12:14:06.004802 25941 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
W20260709 12:14:06.000751 24012 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.509s	user 0.011s	sys 0.000s
W20260709 12:14:05.989356 25896 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.536s	user 0.005s	sys 0.005s
I20260709 12:14:06.024063 25697 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:06.041878 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 729) took 2312 ms. Trace:
W20260709 12:14:06.042910 25812 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.352s	user 0.012s	sys 0.001s
I20260709 12:14:06.043360 25175 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.004300 25779 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:05.998764 25945 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:05.971952 25698 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.008038 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1727) took 2050 ms. Trace:
I20260709 12:14:06.010972 25957 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:06.053066 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:06.075891 24011 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:14:06.051331 25813 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.368s	user 0.009s	sys 0.004s
W20260709 12:14:06.089392 25966 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:06.092339 25852 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.045325 25911 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:05.900264 25913 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.109714 23389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 409) took 3409 ms (client timeout 1995 ms). Trace:
W20260709 12:14:06.115712 23395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 420) took 2199 ms (client timeout 2207 ms). Trace:
W20260709 12:14:06.122689 23389 rpcz_store.cc:269] 0709 12:14:02.700664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.700796 (+   132us) service_pool.cc:224] Handling call
0709 12:14:06.109407 (+3408611us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54001969d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.109595 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163856128}
I20260709 12:14:06.120457 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1767) took 2664 ms. Trace:
I20260709 12:14:06.126199 23318 rpcz_store.cc:276] 0709 12:14:03.456238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.083723 (+627485us) service_pool.cc:224] Handling call
0709 12:14:04.455356 (+371633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.455563 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.455602 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.455671 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:04.455721 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.455792 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.120246 (+1664454us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b5400296150. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.120383 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853423360}
I20260709 12:14:06.062374 24008 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:14:06.123484 23395 rpcz_store.cc:269] 0709 12:14:03.916848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.033878 (+117030us) service_pool.cc:224] Handling call
0709 12:14:06.115558 (+2081680us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001e0fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.115654 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341503872}
I20260709 12:14:06.060854 25959 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:06.103288 25912 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.620s	user 0.008s	sys 0.005s
I20260709 12:14:06.130121 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1776) took 1940 ms. Trace:
I20260709 12:14:06.099922 25698 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.109565 25935 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.002s
I20260709 12:14:06.066401 25697 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.134641 25913 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.080466 25957 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:06.115056 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2187) took 2089 ms (client timeout 1925 ms). Trace:
I20260709 12:14:06.104013 25813 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.079849 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:06.136771 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1709) took 3404 ms. Trace:
I20260709 12:14:06.076356 23461 rpcz_store.cc:276] 0709 12:14:03.957353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.984213 (+ 26860us) service_pool.cc:224] Handling call
0709 12:14:04.091913 (+107700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.132601 (+ 40688us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.132643 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.132686 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:04.143043 (+ 10357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.007627 (+1864584us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b540044b954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:06.007716 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.007943 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88114560}
I20260709 12:14:04.240772 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:06.038450 25888 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.173398 23390 rpcz_store.cc:269] 0709 12:14:04.025947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.390674 (+364727us) service_pool.cc:224] Handling call
0709 12:14:06.114800 (+1724126us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400121954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.114973 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519111040}
I20260709 12:14:06.051031 25812 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:06.105947 25955 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.653s	user 0.004s	sys 0.000s
I20260709 12:14:06.175001 23466 rpcz_store.cc:276] 0709 12:14:02.732879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.733025 (+   146us) service_pool.cc:224] Handling call
0709 12:14:03.723470 (+990445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.724215 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.136565 (+2412350us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b54003edad0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.136709 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2277107072}
I20260709 12:14:06.168687 23315 rpcz_store.cc:276] 0709 12:14:04.189868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.949809 (+759941us) service_pool.cc:224] Handling call
0709 12:14:04.977995 (+ 28186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.981761 (+  3766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.981807 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.982392 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:14:05.508394 (+526002us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:05.508464 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.508537 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.129875 (+621338us) raft_consensus.cc:1685] finished
0709 12:14:06.129900 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.130054 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3295,"raft.queue_time_us":4244,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":15958,"thread_start_us":413868,"threads_started":1,"wal-append.queue_time_us":869273}
I20260709 12:14:06.045908 25896 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.177346 24007 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:06.180371 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:06.175004 25957 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:06.136832 25852 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:06.196662 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:06.200754 25698 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.205681 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:06.207001 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:06.207687 23466 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:06.209486 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:06.209944 23470 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:06.210294 23466 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 2.
I20260709 12:14:06.211871 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:06.218137 25744 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:06.083295 25945 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.178157 25935 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.226131 25935 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.196244 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 724) took 2483 ms. Trace:
I20260709 12:14:06.214067 23461 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:06.175364 25912 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.228283 25945 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:06.208783 24007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 730) took 2478 ms (client timeout 1835 ms). Trace:
I20260709 12:14:06.229705 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:06.247380 25945 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:06.249248 25945 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.249884 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 529) took 3470 ms. Trace:
I20260709 12:14:06.250413 23321 rpcz_store.cc:276] 0709 12:14:02.779129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.035445 (+1256316us) service_pool.cc:224] Handling call
0709 12:14:04.035722 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.276886 (+241164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.276915 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.276955 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:04.276992 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.249532 (+1972540us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54003afa54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:06.249598 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.249802 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554207104}
W20260709 12:14:06.234030 23286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.001s	sys 0.022s
W20260709 12:14:06.254405 23286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.001s	sys 0.022s
I20260709 12:14:06.252048 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1774) took 2062 ms. Trace:
I20260709 12:14:06.258299 24011 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:06.259865 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 740) took 2144 ms (client timeout 2039 ms). Trace:
I20260709 12:14:06.258086 23316 rpcz_store.cc:276] 0709 12:14:04.189360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.715254 (+525894us) service_pool.cc:224] Handling call
0709 12:14:04.724550 (+  9296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.730434 (+  5884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.730489 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.731045 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:14:05.894646 (+1163601us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:05.894689 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.894764 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.251768 (+357004us) raft_consensus.cc:1685] finished
0709 12:14:06.251798 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.251972 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7658,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":1128743,"threads_started":1,"wal-append.queue_time_us":1480854}
I20260709 12:14:06.047411 24010 rpcz_store.cc:276] 0709 12:14:03.729600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.734748 (+  5148us) service_pool.cc:224] Handling call
0709 12:14:04.312153 (+577405us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:04.312365 (+   212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.312916 (+   551us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:04.312957 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.314073 (+  1116us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.620585 (+306512us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:04.630501 (+  9916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.638711 (+  8210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.027934 (+1389223us) raft_consensus.cc:1685] finished
0709 12:14:06.033016 (+  5082us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.041760 (+  8744us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6845,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":9441024,"thread_start_us":275684,"threads_started":1,"wal-append.queue_time_us":432954}
W20260709 12:14:06.291074 24011 rpcz_store.cc:269] 0709 12:14:04.115786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.034173 (+1918387us) service_pool.cc:224] Handling call
0709 12:14:06.259796 (+225623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.272745 23461 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:14:06.292513 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 741) took 1976 ms (client timeout 1863 ms). Trace:
I20260709 12:14:06.293176 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1749) took 1198 ms. Trace:
I20260709 12:14:06.293768 23461 rpcz_store.cc:276] 0709 12:14:05.094847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.250195 (+1155348us) service_pool.cc:224] Handling call
0709 12:14:06.250558 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.292794 (+ 42236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.293102 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.261221 24009 raft_consensus.cc:2368] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
W20260709 12:14:06.276381 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.005s	sys 0.021s
W20260709 12:14:06.297194 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.006s	sys 0.021s
I20260709 12:14:06.297745 23461 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:06.299048 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1751) took 1203 ms. Trace:
I20260709 12:14:06.299577 23461 rpcz_store.cc:276] 0709 12:14:05.095338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.294844 (+1199506us) service_pool.cc:224] Handling call
0709 12:14:06.296677 (+  1833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.298765 (+  2088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.298991 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.298142 25697 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:06.292539 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 509) took 2180 ms (client timeout 2052 ms). Trace:
W20260709 12:14:06.300770 24010 rpcz_store.cc:269] 0709 12:14:04.112119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.292219 (+2180100us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:06.292417 (+   198us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:06.227968 24008 rpcz_store.cc:276] 0709 12:14:03.712589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.725264 (+ 12675us) service_pool.cc:224] Handling call
0709 12:14:06.061603 (+2336339us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:06.061880 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.174381 (+112501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.186053 (+ 11672us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23187200}
I20260709 12:14:06.301873 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:06.302150 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.302690 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 743) took 1970 ms (client timeout 1976 ms). Trace:
W20260709 12:14:06.303076 24010 rpcz_store.cc:269] 0709 12:14:04.332503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.301561 (+1969058us) service_pool.cc:224] Handling call
0709 12:14:06.302630 (+  1069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.304086 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.304847 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 515) took 1583 ms (client timeout 1759 ms). Trace:
I20260709 12:14:06.303083 24008 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 12:14:06.306341 24008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 506) took 2262 ms (client timeout 2301 ms). Trace:
W20260709 12:14:06.306689 24008 rpcz_store.cc:269] 0709 12:14:04.044049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.301882 (+2257833us) service_pool.cc:224] Handling call
0709 12:14:06.306261 (+  4379us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.307626 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.308343 24008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 508) took 2208 ms (client timeout 2393 ms). Trace:
W20260709 12:14:06.277725 23287 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.000s	sys 0.022s
W20260709 12:14:06.309804 23287 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.000s	sys 0.022s
I20260709 12:14:06.309367 25882 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.310950 25882 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.312695 25817 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.309754 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:06.310464 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 725478us
W20260709 12:14:06.314253 23393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1422) took 2765 ms (client timeout 1664 ms). Trace:
I20260709 12:14:06.314306 23394 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
W20260709 12:14:06.314708 23393 rpcz_store.cc:269] 0709 12:14:03.548909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.549046 (+   137us) service_pool.cc:224] Handling call
0709 12:14:06.305401 (+2756355us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:06.314181 (+  8780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5843712}
W20260709 12:14:06.232738 24007 rpcz_store.cc:269] 0709 12:14:03.729968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.734830 (+  4862us) service_pool.cc:224] Handling call
0709 12:14:06.079588 (+2344758us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:06.204000 (+124412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97990912}
W20260709 12:14:06.309283 24008 rpcz_store.cc:269] 0709 12:14:04.099940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.307337 (+2207397us) service_pool.cc:224] Handling call
0709 12:14:06.308289 (+   952us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.321372 25922 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:06.321723 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.322240 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:06.321872 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.323149 24008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 512) took 1614 ms (client timeout 1800 ms). Trace:
W20260709 12:14:06.323484 24007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 513) took 1612 ms (client timeout 1797 ms). Trace:
W20260709 12:14:06.323916 24007 rpcz_store.cc:269] 0709 12:14:04.711236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.321627 (+1610391us) service_pool.cc:224] Handling call
0709 12:14:06.323431 (+  1804us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.324961 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:06.327705 24007 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:06.328065 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:06.330113 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:06.330730 23389 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:06.330617 25957 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.331426 25912 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.325131 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:06.325717 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:06.325943 23316 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:06.325960 25813 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.326938 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:06.327100 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:06.332816 25896 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.346164 24016 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:14:06.440930 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.020s	sys 0.022s
W20260709 12:14:06.527297 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.023s	sys 0.024s
W20260709 12:14:06.373836 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54435.2us
I20260709 12:14:06.434260 23318 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:06.376044 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:06.293006 24011 rpcz_store.cc:269] 0709 12:14:04.316781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.292236 (+1975455us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:06.292414 (+   178us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:06.527920 23389 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.534766 23315 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:06.454592 23433 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.796s	user 0.000s	sys 0.043s
W20260709 12:14:06.535501 23433 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.000s	sys 0.048s
W20260709 12:14:06.543794 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.305192 24010 rpcz_store.cc:269] 0709 12:14:04.721577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.303777 (+1582200us) service_pool.cc:224] Handling call
0709 12:14:06.304794 (+  1017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.538115 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:06.539121 23321 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:06.568312 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 514) took 1855 ms (client timeout 1955 ms). Trace:
W20260709 12:14:06.570327 24011 rpcz_store.cc:269] 0709 12:14:04.713359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.537873 (+1824514us) service_pool.cc:224] Handling call
0709 12:14:06.568230 (+ 30357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.572048 23287 leader_election.cc:400] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:06.572569 23287 leader_election.cc:403] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:06.332955 25957 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.519554 25714 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.496392 25775 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:06.347532 25930 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.620s	user 0.004s	sys 0.009s
I20260709 12:14:06.583459 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:06.473295 25888 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:06.555996 25817 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.565580 25697 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:06.453954 25945 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.556396 25812 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.321893 25922 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:06.537880 25800 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:06.607688 23316 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:14:06.585991 25947 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.067s	user 0.012s	sys 0.000s
I20260709 12:14:06.602710 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.643419 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.643972 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:06.604020 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:06.323527 24008 rpcz_store.cc:269] 0709 12:14:04.709001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.321959 (+1612958us) service_pool.cc:224] Handling call
0709 12:14:06.323087 (+  1128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.645256 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:06.646639 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:06.647883 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:06.608564 23318 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:06.639644 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.588802 25813 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:06.639866 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:06.386565 25956 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.606798 25924 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:06.643899 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.656728 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 744) took 1685 ms. Trace:
I20260709 12:14:06.606027 23319 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:06.589298 25911 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:06.610077 25935 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:06.604549 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.567445 25913 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 12:14:06.654436 23284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.012s	sys 0.029s
I20260709 12:14:06.609661 25957 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:06.608250 23461 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:06.657111 23313 raft_consensus.cc:2410] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:06.657068 24011 rpcz_store.cc:276] 0709 12:14:04.971383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.573227 (+1601844us) service_pool.cc:224] Handling call
0709 12:14:06.656670 (+ 83443us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.655323 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.185s	user 0.015s	sys 0.030s
W20260709 12:14:06.685288 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.215s	user 0.015s	sys 0.030s
W20260709 12:14:06.685609 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 1.21555e+06us
I20260709 12:14:06.686169 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 737) took 2841 ms. Trace:
I20260709 12:14:06.686451 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:06.686501 24011 rpcz_store.cc:276] 0709 12:14:03.844601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.685341 (+2840740us) service_pool.cc:224] Handling call
0709 12:14:06.685701 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.685884 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.685908 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.685947 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:06.685981 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.686025 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.686117 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.658016 25863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.016s	sys 0.025s
W20260709 12:14:06.687512 25863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.017s	sys 0.026s
W20260709 12:14:06.658388 25811 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.005s	sys 0.024s
W20260709 12:14:06.658735 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.016s	sys 0.022s
W20260709 12:14:06.688259 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.016s	sys 0.023s
I20260709 12:14:06.688473 24011 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:06.689026 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 738) took 2843 ms. Trace:
W20260709 12:14:06.651603 23317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.014s	sys 0.025s
W20260709 12:14:06.691170 23317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.015s	sys 0.025s
I20260709 12:14:06.589380 25714 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.660745 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.007s	sys 0.021s
W20260709 12:14:06.682322 23284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.017s	sys 0.031s
I20260709 12:14:06.682663 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.683791 23321 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:06.691820 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1775) took 2502 ms. Trace:
I20260709 12:14:06.609457 25852 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:06.725903 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.007s	sys 0.027s
I20260709 12:14:06.723961 23317 rpcz_store.cc:276] 0709 12:14:04.189624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.907692 (+718068us) service_pool.cc:224] Handling call
0709 12:14:04.908040 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.908508 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:04.958477 (+ 49969us) log.cc:844] Serialized 14 byte log entry
0709 12:14:06.691492 (+1733015us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:06.691543 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:06.691586 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.691648 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.691759 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39568,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":5137,"raft.queue_time_us":105449,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":310,"spinlock_wait_cycles":899968,"thread_start_us":1668778,"threads_started":2,"wal-append.queue_time_us":717186}
I20260709 12:14:06.727098 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1772) took 3145 ms. Trace:
I20260709 12:14:06.689303 24011 rpcz_store.cc:276] 0709 12:14:03.845190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.687988 (+2842798us) service_pool.cc:224] Handling call
0709 12:14:06.688287 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.688816 (+   529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.688973 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.616732 25817 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:06.690867 25678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.012s	sys 0.035s
I20260709 12:14:06.661549 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:06.643072 25947 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.695402 25811 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.005s	sys 0.025s
I20260709 12:14:06.681026 25812 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.727710 23312 rpcz_store.cc:276] 0709 12:14:03.581496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.454573 (+873077us) service_pool.cc:224] Handling call
0709 12:14:04.454981 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:04.484211 (+ 29230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.484240 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:04.493100 (+  8860us) log.cc:844] Serialized 52 byte log entry
0709 12:14:04.775801 (+282701us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:05.866492 (+1090691us) log.cc:844] Serialized 14 byte log entry
0709 12:14:06.726803 (+860311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.726884 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.726925 (+    41us) raft_consensus.cc:1685] finished
0709 12:14:06.726936 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.727046 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15353,"prepare.queue_time_us":15712,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":38,"raft.queue_time_us":29972,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":102148,"thread_start_us":2068204,"threads_started":3,"wal-append.queue_time_us":1036870,"wal-append.run_cpu_time_us":2096,"wal-append.run_wall_time_us":1002102}
I20260709 12:14:06.602087 25775 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.651667 25800 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.696141 25976 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:06.598625 25930 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.692345 24007 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:06.657150 25813 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.728655 25698 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.744789 24011 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:06.952955 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 742) took 2635 ms. Trace:
I20260709 12:14:06.953459 24011 rpcz_store.cc:276] 0709 12:14:04.317275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.731627 (+2414352us) service_pool.cc:224] Handling call
0709 12:14:06.744479 (+ 12852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.952571 (+208092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.952873 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:06.745182 23463 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.010s	sys 0.015s
W20260709 12:14:06.735525 25678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.012s	sys 0.036s
I20260709 12:14:06.730098 25911 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:06.642239 25697 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.692776 25935 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.691332 25957 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.736066 25852 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.966542 25852 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.966583 25957 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:06.967190 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 499) took 2937 ms (client timeout 2067 ms). Trace:
I20260709 12:14:06.737535 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1752) took 1572 ms. Trace:
I20260709 12:14:06.967801 25998 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.967851 23461 rpcz_store.cc:276] 0709 12:14:05.163763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.343012 (+1179249us) service_pool.cc:224] Handling call
0709 12:14:06.575300 (+232288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.695448 (+120148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.736118 (+ 40670us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.968578 25998 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.729705 25912 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:06.970213 25887 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 3.360s	user 0.015s	sys 0.011s
W20260709 12:14:06.781819 25739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.014s	sys 0.014s
I20260709 12:14:06.970800 25998 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:06.971035 25739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.014s	sys 0.014s
W20260709 12:14:06.858747 25877 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.001s	sys 0.027s
I20260709 12:14:06.971359 25998 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:06.858556 24012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.010s	sys 0.018s
I20260709 12:14:06.971623 25911 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:06.972038 24012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.010s	sys 0.018s
I20260709 12:14:06.862849 23469 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:06.972389 24015 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:14:06.972988 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 731) took 3242 ms. Trace:
I20260709 12:14:06.785104 24008 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:06.972863 25911 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.973928 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 735) took 3129 ms. Trace:
I20260709 12:14:06.974144 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1748) took 1879 ms. Trace:
W20260709 12:14:06.859652 25984 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:14:06.737890 25945 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 12:14:06.974329 24008 rpcz_store.cc:276] 0709 12:14:03.844009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.657035 (+2813026us) service_pool.cc:224] Handling call
0709 12:14:06.749622 (+ 92587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.973683 (+224061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.973873 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:06.869558 25977 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:06.971670 25877 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.002s	sys 0.028s
I20260709 12:14:06.976585 25977 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:06.977761 25977 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.808280 25811 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.752789 25812 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.738471 25922 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:06.913878 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 266064us
I20260709 12:14:06.987998 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:06.923784 23467 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.044s	sys 0.110s
I20260709 12:14:06.937345 25817 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.656682 25924 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.870106 25972 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:06.889946 25980 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:14:06.933570 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 427873us
W20260709 12:14:06.888588 25991 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:14:06.954704 23463 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.104s	user 0.010s	sys 0.015s
I20260709 12:14:06.876995 25775 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:06.932889 25661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.011s	sys 0.038s
I20260709 12:14:06.959756 25698 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:06.969810 25912 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.971825 25998 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:06.776575 24010 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:06.974893 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1708) took 4249 ms. Trace:
I20260709 12:14:06.975925 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:07.008875 25980 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:06.980502 24015 rpcz_store.cc:276] 0709 12:14:03.730342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.730484 (+   142us) service_pool.cc:224] Handling call
0709 12:14:06.971865 (+3241381us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:06.972063 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.972736 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.972926 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8174848}
I20260709 12:14:06.985529 24011 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:06.988987 23467 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.044s	sys 0.110s
W20260709 12:14:06.932168 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 245753us
W20260709 12:14:06.958948 24007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 745) took 1911 ms (client timeout 1553 ms). Trace:
W20260709 12:14:07.031375 24007 rpcz_store.cc:269] 0709 12:14:05.047922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.324670 (+1276748us) service_pool.cc:224] Handling call
0709 12:14:06.956072 (+631402us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:07.002391 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.011121 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.039456 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 511) took 2332 ms. Trace:
I20260709 12:14:07.039742 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1742) took 1970 ms. Trace:
I20260709 12:14:07.011075 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1740) took 2719 ms. Trace:
I20260709 12:14:07.039958 24011 rpcz_store.cc:276] 0709 12:14:04.707532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.954450 (+2246918us) service_pool.cc:224] Handling call
0709 12:14:06.985187 (+ 30737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.030520 (+ 45333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.039391 (+  8871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:07.042735 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1697) took 5163 ms. Trace:
I20260709 12:14:07.043141 23462 rpcz_store.cc:276] 0709 12:14:01.879352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.883009 (+  3657us) service_pool.cc:224] Handling call
0709 12:14:01.883351 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.655009 (+1771658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.042474 (+3387465us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54001efad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:07.042675 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4073228160}
I20260709 12:14:07.044260 25698 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.986729 25714 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:06.959611 25976 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:06.998693 25924 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:07.048064 25697 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:07.048638 24016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 507) took 2960 ms (client timeout 1746 ms). Trace:
I20260709 12:14:07.020313 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:07.049090 24016 rpcz_store.cc:269] 0709 12:14:04.088306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.190039 (+101733us) service_pool.cc:224] Handling call
0709 12:14:07.048433 (+2858394us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54006eebd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:07.048557 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4748859776}
I20260709 12:14:07.007534 25922 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:07.050110 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:07.050489 25813 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:06.988554 25811 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:07.051311 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.051998 24016 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:07.052582 25811 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.052677 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:07.053081 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:07.053912 23396 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:07.054164 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:07.054667 25947 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.023859 23465 rpcz_store.cc:276] 0709 12:14:02.725942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.726082 (+   140us) service_pool.cc:224] Handling call
0709 12:14:02.748333 (+ 22251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.748575 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.748608 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:02.928566 (+179958us) log.cc:844] Serialized 26 byte log entry
0709 12:14:03.218445 (+289879us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:03.218498 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.218604 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.969633 (+3751029us) spinlock_profiling.cc:243] Waited 8.51 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:06.974597 (+  4964us) raft_consensus.cc:1685] finished
0709 12:14:06.974618 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.974817 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19580672,"thread_start_us":164951,"threads_started":1,"wal-append.queue_time_us":514016}
I20260709 12:14:06.974527 23469 rpcz_store.cc:276] 0709 12:14:05.094574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.434433 (+339859us) service_pool.cc:224] Handling call
0709 12:14:05.434872 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.972621 (+1537749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.973953 (+  1332us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:06.974092 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3283652480}
I20260709 12:14:07.015651 25972 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:07.044262 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:07.045699 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.062633 24015 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:07.011556 25961 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:07.046339 23362 connection.cc:256] RPC call timeout handler was delayed by -1.0830909449998671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 12:14:07.045552 23463 rpcz_store.cc:276] 0709 12:14:04.291176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.441419 (+150243us) service_pool.cc:224] Handling call
0709 12:14:04.441806 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.442007 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:14:05.356801 (+914794us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.010666 (+1653865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.010731 (+    65us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:07.010788 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.010875 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.011009 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1276049,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":173577,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":2528570,"threads_started":3,"wal-append.queue_time_us":874471}
I20260709 12:14:07.047143 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.048763 25976 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:07.050202 24007 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:07.016654 25997 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.054858 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:06.967677 24009 rpcz_store.cc:269] 0709 12:14:04.029714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.619611 (+589897us) service_pool.cc:224] Handling call
0709 12:14:06.966936 (+2347325us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540049eb54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:06.967103 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489707392}
I20260709 12:14:07.054978 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:07.055302 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:07.019057 25661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.011s	sys 0.039s
I20260709 12:14:07.056281 24016 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:07.056839 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:07.025676 25996 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.040231 23467 rpcz_store.cc:276] 0709 12:14:05.069137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.069261 (+   124us) service_pool.cc:224] Handling call
0709 12:14:05.069643 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:05.898833 (+829190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.925534 (+ 26701us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:05.959248 (+ 33714us) log.cc:844] Serialized 52 byte log entry
0709 12:14:07.030656 (+1071408us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:14:07.039146 (+  8490us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.039394 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.039482 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.039527 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:07.039540 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.039685 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23201,"raft.queue_time_us":837266,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":9694,"thread_start_us":1747362,"threads_started":2,"wal-append.queue_time_us":939397}
I20260709 12:14:07.027590 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:07.064395 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.915437 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.915993 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.919495 24014 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:07.057014 25800 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.065346 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 733) took 3216 ms. Trace:
I20260709 12:14:07.055449 23461 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:07.920987 24010 rpcz_store.cc:276] 0709 12:14:03.843380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.604982 (+2761602us) service_pool.cc:224] Handling call
0709 12:14:06.696519 (+ 91537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.028305 (+331786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.060058 (+ 31753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:07.151361 25882 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.922554 23396 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:14:07.917804 23362 connection.cc:256] RPC call timeout handler was delayed by -1.0695299159999649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 12:14:07.102468 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.918134 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.919396 25848 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:07.915506 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.917130 24008 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
W20260709 12:14:07.748371 25977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.029s	sys 0.048s
I20260709 12:14:07.101480 23395 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:07.118613 25947 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.167717 26000 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.920418 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:07.901340 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.989604 23387 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
W20260709 12:14:07.921676 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2247) took 1606 ms (client timeout 1672 ms). Trace:
I20260709 12:14:07.164552 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.067826 23465 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:08.068795 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 446) took 1494 ms. Trace:
I20260709 12:14:07.103844 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1750) took 2008 ms. Trace:
W20260709 12:14:08.071355 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 417) took 4180 ms (client timeout 2068 ms). Trace:
W20260709 12:14:08.073289 23982 connection.cc:256] RPC call timeout handler was delayed by -2.6204132719999507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2009s for a response
W20260709 12:14:08.073587 25977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.030s	sys 0.050s
W20260709 12:14:08.073460 23387 rpcz_store.cc:269] 0709 12:14:03.888921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.889065 (+   144us) service_pool.cc:224] Handling call
0709 12:14:07.056710 (+3167645us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:08.069549 (+1012839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11230976}
W20260709 12:14:07.196007 25933 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 2.427s	user 0.015s	sys 0.010s
I20260709 12:14:08.069259 23465 rpcz_store.cc:276] 0709 12:14:06.574044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.114587 (+540543us) service_pool.cc:224] Handling call
0709 12:14:08.068667 (+954080us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:07.728842 25950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.154s	user 0.007s	sys 0.007s
W20260709 12:14:07.153008 25925 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 2.463s	user 0.021s	sys 0.017s
W20260709 12:14:07.659650 25922 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.609s	user 0.009s	sys 0.002s
W20260709 12:14:07.655236 25924 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.000s
I20260709 12:14:07.110606 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:08.062711 23461 rpcz_store.cc:269] 0709 12:14:06.314836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.969141 (+654305us) service_pool.cc:224] Handling call
0709 12:14:07.921405 (+952264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:07.046272 25980 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:07.186091 26001 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:07.437039 25899 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:08.085556 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.071717 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 653) took 1028 ms. Trace:
W20260709 12:14:07.901369 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 913221us
I20260709 12:14:07.427567 25930 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.175793 25812 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:07.174649 25998 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.078703 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 651) took 1035 ms. Trace:
I20260709 12:14:08.079941 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:07.424392 25934 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.103600 24011 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:08.104058 25956 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.104444 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 648) took 1062 ms. Trace:
I20260709 12:14:07.177057 25935 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.106994 23462 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:08.107180 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:08.110577 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.111696 23463 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:08.112213 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2250) took 1069 ms. Trace:
I20260709 12:14:08.112555 23463 rpcz_store.cc:276] 0709 12:14:07.042398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.109777 (+1067379us) service_pool.cc:224] Handling call
0709 12:14:08.112136 (+  2359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.104935 24011 rpcz_store.cc:276] 0709 12:14:07.041630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.046312 (+  4682us) service_pool.cc:224] Handling call
0709 12:14:08.104319 (+1058007us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.121423 24015 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:08.073976 25961 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.109012 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:08.125545 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1432) took 1045 ms. Trace:
I20260709 12:14:08.126052 23395 rpcz_store.cc:276] 0709 12:14:07.051893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.055020 (+  3127us) service_pool.cc:224] Handling call
0709 12:14:07.101191 (+ 46171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.074789 (+973598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.097504 (+ 22715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.127728 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:08.128578 23395 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:08.129655 24014 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:08.130262 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 660) took 1085 ms. Trace:
I20260709 12:14:08.095830 23466 rpcz_store.cc:276] 0709 12:14:05.095088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.252033 (+1156945us) service_pool.cc:224] Handling call
0709 12:14:06.252432 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.252928 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.252980 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.253585 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.709192 (+455607us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:06.709249 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.709342 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.103539 (+394197us) raft_consensus.cc:1685] finished
0709 12:14:07.103566 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.103756 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8437,"thread_start_us":455208,"threads_started":1,"wal-append.queue_time_us":480399}
I20260709 12:14:08.130551 24014 rpcz_store.cc:276] 0709 12:14:07.044814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.935291 (+890477us) service_pool.cc:224] Handling call
0709 12:14:08.130203 (+194912us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.090695 24009 rpcz_store.cc:276] 0709 12:14:07.043115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.917506 (+874391us) service_pool.cc:224] Handling call
0709 12:14:08.071612 (+154106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.139819 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 656) took 1096 ms. Trace:
I20260709 12:14:08.105228 25947 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.144160 25930 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.144404 24010 rpcz_store.cc:276] 0709 12:14:07.043742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.922859 (+879117us) service_pool.cc:224] Handling call
0709 12:14:08.139748 (+216889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.104943 25956 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.113116 26000 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.152925 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1438) took 1092 ms. Trace:
I20260709 12:14:08.100680 24008 rpcz_store.cc:276] 0709 12:14:07.042511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.046861 (+  4350us) service_pool.cc:224] Handling call
0709 12:14:08.074888 (+1028027us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:08.151219 23462 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 445) took 1580 ms (client timeout 2058 ms). Trace:
I20260709 12:14:08.146313 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:08.153367 23388 rpcz_store.cc:276] 0709 12:14:07.055670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.078488 (+1022818us) service_pool.cc:224] Handling call
0709 12:14:08.148381 (+ 69893us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:08.155036 23462 rpcz_store.cc:269] 0709 12:14:06.565765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.110235 (+544470us) service_pool.cc:224] Handling call
0709 12:14:08.140719 (+1030484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.159864 25812 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:08.161002 25812 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.164948 25812 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.077353 25996 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.144997 25930 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.167567 25956 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:08.158880 25950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.557s	user 0.011s	sys 0.011s
I20260709 12:14:08.100795 25848 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.094452 25997 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.179991 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1747) took 3085 ms. Trace:
I20260709 12:14:08.180073 25930 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:08.176564 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66368.6us
I20260709 12:14:07.175079 25972 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.187762 25947 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.187902 25997 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.174678 26001 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.180955 23470 rpcz_store.cc:276] 0709 12:14:05.094307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.098526 (+  4219us) service_pool.cc:224] Handling call
0709 12:14:05.098838 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.210510 (+1111672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.179473 (+1968963us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002a08d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:08.179781 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2554756864}
I20260709 12:14:08.180032 25996 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.195510 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:08.196539 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:08.192890 23318 raft_consensus.cc:2376] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
I20260709 12:14:08.198684 23466 raft_consensus.cc:2368] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:08.200744 25956 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.181674 26009 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:08.201157 25916 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 3.782s	user 0.028s	sys 0.018s
I20260709 12:14:08.195597 26001 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.205765 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 454) took 1333 ms. Trace:
I20260709 12:14:08.208050 23466 rpcz_store.cc:276] 0709 12:14:06.869443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.131259 (+1261816us) service_pool.cc:224] Handling call
0709 12:14:08.203149 (+ 71890us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.216003 25848 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.213335 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:08.227710 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 146197us
I20260709 12:14:08.226084 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1800) took 1202 ms. Trace:
I20260709 12:14:08.231856 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 726) took 4518 ms. Trace:
I20260709 12:14:08.204401 25924 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.229790 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:08.232643 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1780) took 4041 ms. Trace:
I20260709 12:14:08.234972 23320 rpcz_store.cc:276] 0709 12:14:04.190916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.168007 (+977091us) service_pool.cc:224] Handling call
0709 12:14:05.168378 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.871006 (+2702628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.871108 (+   102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.871223 (+   115us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:07.871310 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.231998 (+360688us) spinlock_profiling.cc:243] Waited 2.7 s on lock 0x7b5400149454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:08.232073 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.232359 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6213758208}
I20260709 12:14:08.236281 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.238138 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:08.239887 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:08.242527 26000 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.244853 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:08.245363 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.245702 25924 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.246102 23461 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:08.246223 23982 connection.cc:256] RPC call timeout handler was delayed by -1.843608685999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 12:14:08.246747 23982 connection.cc:256] RPC call timeout handler was delayed by -1.625733615999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 12:14:08.246783 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 455) took 1073 ms. Trace:
W20260709 12:14:08.247223 23982 connection.cc:256] RPC call timeout handler was delayed by -1.6167336159999195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
I20260709 12:14:08.247294 23461 rpcz_store.cc:276] 0709 12:14:07.173210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.244372 (+1071162us) service_pool.cc:224] Handling call
0709 12:14:08.246713 (+  2341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.249541 25920 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.228971 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:08.247614 23982 connection.cc:256] RPC call timeout handler was delayed by -1.6167336159999195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
I20260709 12:14:08.240432 23467 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:08.270219 23462 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:08.229847 23318 rpcz_store.cc:276] 0709 12:14:07.010528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.011164 (+   636us) service_pool.cc:224] Handling call
0709 12:14:08.210737 (+1199573us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:08.276381 23982 connection.cc:256] RPC call timeout handler was delayed by -1.4916336159999446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 12:14:08.277742 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2249) took 1230 ms. Trace:
I20260709 12:14:08.272987 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2248) took 1231 ms. Trace:
I20260709 12:14:08.279047 23462 rpcz_store.cc:276] 0709 12:14:07.041839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.190850 (+1149011us) service_pool.cc:224] Handling call
0709 12:14:08.272903 (+ 82053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.248699 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.278286 23467 rpcz_store.cc:276] 0709 12:14:07.042119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.235697 (+1193578us) service_pool.cc:224] Handling call
0709 12:14:08.271971 (+ 36274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.282892 23395 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
W20260709 12:14:08.284323 23982 connection.cc:256] RPC call timeout handler was delayed by -1.4709336159999111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 12:14:08.280943 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.233795 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.301901 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.232424 24013 rpcz_store.cc:276] 0709 12:14:03.713184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.717375 (+  4191us) service_pool.cc:224] Handling call
0709 12:14:04.329840 (+612465us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:04.330511 (+   671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.330833 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:04.443730 (+112897us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.213293 (+3769563us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b18003117a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:14:08.230918 (+ 17625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.231047 (+   129us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.231233 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.231383 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.231642 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19963,"prepare.queue_time_us":167614,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":3719,"raft.queue_time_us":3624543,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":249,"spinlock_wait_cycles":21539968,"thread_start_us":264358,"threads_started":2,"wal-append.queue_time_us":293107,"wal-append.run_cpu_time_us":2586,"wal-append.run_wall_time_us":1568533}
I20260709 12:14:08.310935 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1734) took 4220 ms. Trace:
I20260709 12:14:08.311508 23468 rpcz_store.cc:276] 0709 12:14:04.090790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.090922 (+   132us) service_pool.cc:224] Handling call
0709 12:14:04.091263 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.091632 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.091658 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.097398 (+  5740us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.201688 (+104290us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:04.201732 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.201797 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.200060 (+3998263us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:08.310422 (+110362us) raft_consensus.cc:1685] finished
0709 12:14:08.310447 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.310733 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2318,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":9992,"spinlock_wait_cycles":34776832,"thread_start_us":64901,"threads_started":1,"wal-append.queue_time_us":244912}
I20260709 12:14:08.323246 23318 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:08.323858 24007 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:08.324232 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.324568 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 658) took 1280 ms. Trace:
I20260709 12:14:08.325769 25897 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.327189 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:08.327966 23318 raft_consensus.cc:2376] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
I20260709 12:14:08.328574 23468 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:08.327225 24007 rpcz_store.cc:276] 0709 12:14:07.044242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.919923 (+875681us) service_pool.cc:224] Handling call
0709 12:14:08.324480 (+404557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.329617 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 442) took 1790 ms. Trace:
I20260709 12:14:08.330018 23468 rpcz_store.cc:276] 0709 12:14:06.538787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.327803 (+1789016us) service_pool.cc:224] Handling call
0709 12:14:08.328251 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.329174 (+   923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.329531 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.331848 23468 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:08.332399 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 443) took 1793 ms. Trace:
I20260709 12:14:08.332803 23468 rpcz_store.cc:276] 0709 12:14:06.539053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.330825 (+1791772us) service_pool.cc:224] Handling call
0709 12:14:08.331334 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.332159 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.332354 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.334267 23320 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:14:08.335425 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 110086us
I20260709 12:14:08.337556 26010 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:14:08.250303 25920 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.332299 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:08.286408 26009 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.358078 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 725) took 4644 ms. Trace:
I20260709 12:14:08.358597 26009 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.359014 24012 rpcz_store.cc:276] 0709 12:14:03.712871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.725322 (+ 12451us) service_pool.cc:224] Handling call
0709 12:14:04.320359 (+595037us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:04.320579 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.320865 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:06.039311 (+1718446us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.356703 (+2317392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.356913 (+   210us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:08.357148 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.357392 (+   244us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.357686 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1265342,"raft.queue_time_us":1267433,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":170,"spinlock_wait_cycles":5530752,"thread_start_us":1039784,"threads_started":2,"wal-append.queue_time_us":847683,"wal-append.run_cpu_time_us":1280,"wal-append.run_wall_time_us":1058249}
I20260709 12:14:08.341185 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:08.359345 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1733) took 4273 ms. Trace:
I20260709 12:14:08.369601 25870 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:08.369818 23318 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:14:08.372895 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.374246 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.369830 23470 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:08.368778 23461 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:08.359604 25920 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.381227 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 448) took 1762 ms. Trace:
W20260709 12:14:08.381865 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.366164 25950 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:14:08.381556 23470 rpcz_store.cc:276] 0709 12:14:06.616936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.210957 (+1594021us) service_pool.cc:224] Handling call
0709 12:14:08.378693 (+167736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.388579 23466 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:08.389133 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 452) took 1645 ms. Trace:
I20260709 12:14:08.389557 23466 rpcz_store.cc:276] 0709 12:14:06.744112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.213654 (+1469542us) service_pool.cc:224] Handling call
0709 12:14:08.389060 (+175406us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.390286 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.391554 26000 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:08.393283 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 194636us
I20260709 12:14:08.394780 25961 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.385139 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:08.367687 23464 rpcz_store.cc:276] 0709 12:14:04.086289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.092442 (+  6153us) service_pool.cc:224] Handling call
0709 12:14:04.115953 (+ 23511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.116381 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.116406 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.116854 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.438437 (+321583us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:04.438478 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.438564 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.152386 (+2713822us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:08.280771 (+1128385us) raft_consensus.cc:1685] finished
0709 12:14:08.286215 (+  5444us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.359189 (+ 72974us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9,"raft.queue_time_us":88,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5776,"spinlock_wait_cycles":12385152,"thread_start_us":320775,"threads_started":1,"wal-append.queue_time_us":511893}
I20260709 12:14:08.404939 25661 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:14:08.411180 25661 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:08.411716 25661 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:08.415604 25912 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.398897 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:08.420202 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.397842 26009 raft_consensus.cc:2764] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:14:08.421840 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:08.424319 23465 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:14:08.425760 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 101743us
I20260709 12:14:08.426723 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:08.398435 25950 consensus_queue.cc:1243] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:14:08.423815 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 441) took 1882 ms. Trace:
I20260709 12:14:08.430199 23462 rpcz_store.cc:276] 0709 12:14:06.538469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.349573 (+1811104us) service_pool.cc:224] Handling call
0709 12:14:08.377406 (+ 27833us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.420148 (+ 42742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.420184 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.420226 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.420260 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.420318 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.420431 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.431735 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:08.418958 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1760) took 1380 ms. Trace:
I20260709 12:14:08.432787 23461 rpcz_store.cc:276] 0709 12:14:07.038830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.248248 (+1209418us) service_pool.cc:224] Handling call
0709 12:14:08.408954 (+160706us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.436201 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1753) took 1399 ms. Trace:
I20260709 12:14:08.436669 23461 rpcz_store.cc:276] 0709 12:14:07.036998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.435052 (+1398054us) service_pool.cc:224] Handling call
0709 12:14:08.435548 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.435849 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.435877 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.435926 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:08.435966 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.436028 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.436144 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.467406 26016 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.439436 25980 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.477785 26016 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.479315 23466 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:14:08.479451 25980 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.479693 23466 pending_rounds.cc:85] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 7
I20260709 12:14:08.480046 23466 pending_rounds.cc:107] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:14:08.480347 23466 raft_consensus.cc:2889] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:08.483237 23461 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:08.441718 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:08.442368 26000 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:08.444231 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1754) took 1406 ms. Trace:
I20260709 12:14:08.445701 25882 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:08.446019 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 447) took 1863 ms. Trace:
I20260709 12:14:08.487188 23467 rpcz_store.cc:276] 0709 12:14:06.582881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.400873 (+1817992us) service_pool.cc:224] Handling call
0709 12:14:08.401269 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.429810 (+ 28541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.429842 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.445670 (+ 15828us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.445736 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.445804 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.445940 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.447269 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 453) took 1641 ms. Trace:
I20260709 12:14:08.486944 23469 rpcz_store.cc:276] 0709 12:14:07.037278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.442948 (+1405670us) service_pool.cc:224] Handling call
0709 12:14:08.443315 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.443727 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.443753 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.443808 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:08.443850 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.443904 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.444034 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.447697 25943 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:14:08.450557 23394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2191) took 4409 ms (client timeout 1628 ms). Trace:
W20260709 12:14:08.458012 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 117072us
W20260709 12:14:08.494499 23394 rpcz_store.cc:269] 0709 12:14:04.040787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.107843 (+ 67056us) service_pool.cc:224] Handling call
0709 12:14:08.450159 (+4342316us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54002e0c54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:08.450364 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4931708160}
I20260709 12:14:08.459949 25942 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:08.497241 26000 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:08.497735 26000 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:08.498044 25950 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.498744 25950 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.461714 25920 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:08.462208 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 449) took 1775 ms (client timeout 2230 ms). Trace:
W20260709 12:14:08.499949 23465 rpcz_store.cc:269] 0709 12:14:06.655008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.417168 (+1762160us) service_pool.cc:224] Handling call
0709 12:14:08.430068 (+ 12900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.463605 23982 leader_election.cc:400] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6.
I20260709 12:14:08.465404 26013 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:08.467298 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:08.467437 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:08.467808 26014 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.468225 25922 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:08.468487 25927 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:08.470057 25897 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:08.470896 23317 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:08.471906 25976 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.518576 23392 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:08.518965 25976 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.472147 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:08.474267 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 450) took 1782 ms. Trace:
I20260709 12:14:08.521982 23464 rpcz_store.cc:276] 0709 12:14:06.691372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.406515 (+1715143us) service_pool.cc:224] Handling call
0709 12:14:08.417886 (+ 11371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.473925 (+ 56039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.473950 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.474011 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:08.474041 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.474090 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.474194 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:08.475127 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1759) took 1436 ms (client timeout 1625 ms). Trace:
I20260709 12:14:08.476421 25734 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.412990 25661 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:08.523083 23463 rpcz_store.cc:269] 0709 12:14:07.038570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.113278 (+1074708us) service_pool.cc:224] Handling call
0709 12:14:08.390178 (+276900us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:08.474846 (+ 84668us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:08.475019 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102097024}
W20260709 12:14:08.476825 23979 connection.cc:256] RPC call timeout handler was delayed by -2.261766433000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:14:08.480774 26016 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.480870 25994 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:08.481714 25660 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:08.483292 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1756) took 1445 ms. Trace:
I20260709 12:14:08.506253 26013 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.440732 25912 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:08.532230 26014 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.491408 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:08.492487 23393 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:08.492648 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51126.1us
I20260709 12:14:08.492763 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1758) took 1454 ms. Trace:
I20260709 12:14:08.488562 23470 rpcz_store.cc:276] 0709 12:14:06.806267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.420756 (+1614489us) service_pool.cc:224] Handling call
0709 12:14:08.446652 (+ 25896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.446918 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.446943 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.447002 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.447041 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.447092 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.447206 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.498178 26000 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:08.461059 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1755) took 1423 ms. Trace:
I20260709 12:14:08.542230 23462 rpcz_store.cc:276] 0709 12:14:07.037551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.458212 (+1420661us) service_pool.cc:224] Handling call
0709 12:14:08.458840 (+   628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.459312 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.459338 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.460711 (+  1373us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.460783 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.460856 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.460974 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.515467 25681 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:14:08.517654 23320 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.534835 23469 rpcz_store.cc:276] 0709 12:14:07.038332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.491589 (+1453257us) service_pool.cc:224] Handling call
0709 12:14:08.491925 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.492361 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.492403 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.492488 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:08.492537 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.492598 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.492710 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5539,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109}
I20260709 12:14:08.518368 23312 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:08.528985 23466 rpcz_store.cc:276] 0709 12:14:07.037824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.478555 (+1440731us) service_pool.cc:224] Handling call
0709 12:14:08.478898 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.482960 (+  4062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.483201 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:08.440639 25981 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 1.573s	user 0.013s	sys 0.015s
I20260709 12:14:08.527092 23315 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:08.532476 25358 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.524518 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1757) took 1486 ms. Trace:
I20260709 12:14:08.542876 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1433) took 1489 ms. Trace:
I20260709 12:14:08.533190 25912 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.535439 26013 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:08.550482 23393 rpcz_store.cc:276] 0709 12:14:07.052210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.055096 (+  2886us) service_pool.cc:224] Handling call
0709 12:14:08.491894 (+1436798us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b5400330aa8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af4515f4 00007ff0ae63a227 00007ff0ae639ab8 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:08.541357 (+ 49463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86337408}
I20260709 12:14:08.562201 25358 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.550143 23461 rpcz_store.cc:276] 0709 12:14:07.038075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.482607 (+1444532us) service_pool.cc:224] Handling call
0709 12:14:08.482989 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.488813 (+  5824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.513155 (+ 24342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.562106 25920 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.563942 25681 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:08.564411 25681 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:08.565619 25920 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:08.565817 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 102437us
I20260709 12:14:08.566735 25702 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:08.567200 25702 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:08.568491 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.569257 24007 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:08.566679 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.568195 25702 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.579000 23320 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:08.536331 25897 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:14:08.571004 26014 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.581341 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:08.585250 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 93126.6us
I20260709 12:14:08.553298 25976 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.587707 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 657) took 1456 ms. Trace:
I20260709 12:14:08.588284 23318 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
W20260709 12:14:08.588528 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.590062 23467 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:08.593196 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 456) took 1419 ms. Trace:
I20260709 12:14:08.596518 23467 rpcz_store.cc:276] 0709 12:14:07.173711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.512850 (+1339139us) service_pool.cc:224] Handling call
0709 12:14:08.578381 (+ 65531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.592935 (+ 14554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.593136 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.595276 25927 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.601342 25848 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:08.601806 25848 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.602383 25848 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.602662 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:08.588110 24016 rpcz_store.cc:276] 0709 12:14:07.044002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.951150 (+907148us) service_pool.cc:224] Handling call
0709 12:14:07.951844 (+   694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.244087 (+292243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.244121 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:08.249645 (+  5524us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:08.250159 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:14:08.369082 (+118923us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.369123 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.369192 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.485016 (+115824us) raft_consensus.cc:1685] finished
0709 12:14:08.485040 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.500594 (+ 15554us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":291326,"raft.queue_time_us":296470,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"spinlock_wait_cycles":8658688,"thread_start_us":116465,"threads_started":1,"wal-append.queue_time_us":224396}
I20260709 12:14:08.603768 23313 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:14:08.604077 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.554219 25734 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.609798 24012 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:08.610460 25734 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:08.596781 25913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.814s	user 0.006s	sys 0.049s
I20260709 12:14:08.601596 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:08.611763 25913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.882s	user 0.006s	sys 0.050s
I20260709 12:14:08.599856 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 457) took 1425 ms. Trace:
I20260709 12:14:08.614946 23465 rpcz_store.cc:276] 0709 12:14:07.174044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.526453 (+1352409us) service_pool.cc:224] Handling call
0709 12:14:08.571897 (+ 45444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.595506 (+ 23609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.595534 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.599647 (+  4113us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:08.599681 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.599728 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.599812 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.601049 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:08.613250 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.626190 24015 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:08.590447 25922 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.637655 25899 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:08.595290 26016 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.591161 25920 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.596302 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.598142 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:08.652108 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56956.2us
I20260709 12:14:08.653236 25702 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:08.654206 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 87626.3us
W20260709 12:14:08.655076 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73902.5us
I20260709 12:14:08.640568 25661 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.578724 25681 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:08.662761 23436 leader_election.cc:336] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.513s (SENT)
I20260709 12:14:08.645174 26015 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.669799 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:08.601706 25927 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.675870 24007 raft_consensus.cc:2368] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6.
W20260709 12:14:08.677485 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 104369us
I20260709 12:14:08.641229 26001 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.649895 26010 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:08.679803 25734 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:08.685588 25734 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:08.685703 25681 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.686029 25734 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.696581 25862 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:14:08.696182 25734 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.671177 25920 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.715366 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.666416 25922 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.667766 26016 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.729966 26023 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:14:08.731778 26023 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:08.732277 26023 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:08.738592 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:08.740671 26020 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.742695 25927 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.743171 26001 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.745816 26011 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:08.746284 26011 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:08.747416 26011 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:08.748596 26011 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:08.748858 25937 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:08.758774 24011 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.712388 25950 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:08.773422 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 114522us
W20260709 12:14:08.784858 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47363.9us
W20260709 12:14:08.794579 23436 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.535s (SENT)
I20260709 12:14:08.795193 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:08.795392 25942 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:08.795791 25942 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:08.796228 25942 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.797571 25927 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:08.797935 25993 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:08.798441 25993 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:08.799535 25993 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.801389 26000 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.801829 25950 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.803964 26013 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.807766 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:08.808234 25734 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.808580 24014 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:08.810808 25912 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.801196 25993 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:08.806684 26000 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:08.827384 23465 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:08.828766 23461 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:08.808413 26013 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:08.735502 26010 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:08.850682 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.856177 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:08.857010 23315 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.857928 24014 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:08.837302 26010 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:08.865775 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 186455us
W20260709 12:14:08.877967 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:08.869163 23436 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 2.169s (SENT) [suppressed 41 similar messages]
I20260709 12:14:08.879146 23465 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:08.887838 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.905578 24015 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:08.917366 23317 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:08.868896 26023 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.878144 25870 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.922016 26010 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.924010 26010 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:08.859236 26000 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.949369 25920 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:08.951272 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:08.952253 23320 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:08.956437 25922 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:08.957233 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.957814 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:08.958055 25884 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:14:08.959412 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:08.960459 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:08.961318 26020 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:08.961802 25912 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:08.962146 26020 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:08.962298 25912 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:08.962672 26020 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:08.962808 25912 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:08.963124 26020 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:08.970197 26016 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:08.971700 26019 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.370s	user 0.003s	sys 0.000s
I20260709 12:14:08.971993 26020 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:08.974174 26020 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:08.949214 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 104540us
W20260709 12:14:08.979040 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.986771 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:09.009987 26001 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.012567 25913 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.016948 26001 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.018193 26000 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:09.023226 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.030284 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 391409us
I20260709 12:14:09.031169 24011 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:09.032416 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:09.030233 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:09.025962 26020 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:09.025631 25980 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.057129 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:09.063752 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 230630us
I20260709 12:14:09.017135 25913 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.060087 25937 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:09.096761 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:09.089607 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62254.4us
I20260709 12:14:09.092948 23318 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:09.064742 25980 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.092466 25937 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.126780 23360 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:09.130244 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:09.145272 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.126614 25924 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:14:09.151873 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.135874 26001 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.132891 26013 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:09.170378 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 85003.7us
W20260709 12:14:09.170529 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54690.5us
I20260709 12:14:09.149827 25924 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:09.178579 23315 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:09.154368 25993 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:09.186278 26015 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.004s
I20260709 12:14:09.170077 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:09.186265 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:09.181109 25924 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:09.194202 25848 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.591s	user 0.013s	sys 0.004s
W20260709 12:14:09.235896 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.242569 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56411us
W20260709 12:14:09.203788 25661 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.006s
I20260709 12:14:09.216475 26015 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:09.200227 26024 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:09.191906 25948 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:14:09.202921 26000 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.180446 25937 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.240988 25972 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:09.250141 25993 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:14:09.256055 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86052.9us
I20260709 12:14:09.237893 25358 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.243093 25848 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.261644 25942 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.284228 25913 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.285950 25942 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.286223 25937 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.288003 25884 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:09.288194 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 444) took 2748 ms. Trace:
I20260709 12:14:09.288709 23468 rpcz_store.cc:276] 0709 12:14:06.539328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.364752 (+1825424us) service_pool.cc:224] Handling call
0709 12:14:08.419975 (+ 55223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.431924 (+ 11949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.431952 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:08.728565 (+296613us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:08.729176 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:14:08.989932 (+260756us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:08.989978 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.990060 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.287855 (+297795us) raft_consensus.cc:1685] finished
0709 12:14:09.287873 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.288115 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":392,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":172,"spinlock_wait_cycles":131179904,"thread_start_us":251855,"threads_started":1,"wal-append.queue_time_us":471558}
W20260709 12:14:09.293797 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1428) took 4854 ms (client timeout 2519 ms). Trace:
W20260709 12:14:09.295140 26040 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:14:09.296710 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:09.298079 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:09.296962 26036 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:14:09.306314 24010 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:09.307205 25922 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:09.307597 25922 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.308001 25922 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:09.311679 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:09.294495 23392 rpcz_store.cc:269] 0709 12:14:04.439177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.384749 (+945572us) service_pool.cc:224] Handling call
0709 12:14:09.279354 (+3894605us) spinlock_profiling.cc:243] Waited 3.13 s on lock 0x7b54002c8654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:09.293678 (+ 14324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7188898176}
I20260709 12:14:09.283103 25972 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.314334 26016 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:09.315059 25972 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.316196 25972 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.317919 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:09.321981 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.322299 25899 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:09.322724 25899 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.322831 23461 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:09.323091 25899 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.323547 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.324154 23462 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:09.324247 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.324848 23468 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:09.325642 25980 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.325736 25884 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.326143 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:09.326786 25884 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.327131 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.327207 26033 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:09.328930 26033 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:09.322005 25358 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:14:09.330101 26024 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.331498 26024 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.338919 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.339684 23461 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.344211 25848 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.288164 25973 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.314067 25870 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:09.361716 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 101884us
W20260709 12:14:09.365408 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.366058 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.366051 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.283469 25993 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:09.371286 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.371519 23466 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:14:09.373734 25912 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.374886 25993 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.345607 26015 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:09.367262 25973 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.371454 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:09.378636 23313 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:09.381726 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:09.372123 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.370949 25980 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.385695 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.389346 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.392124 24013 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:09.390226 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.399911 24016 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:09.406704 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.414333 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.414954 24014 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:14:09.417660 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 91633.7us
I20260709 12:14:09.418540 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.419618 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.412967 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.421734 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.422096 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.386140 23317 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:09.430280 23319 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:09.431324 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:09.431952 23319 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:09.432971 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:09.433681 23316 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:09.433772 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.435635 25913 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:09.436074 25913 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.436506 25913 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:09.441805 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.444475 25973 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.445089 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.445945 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.445749 23393 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:09.421396 23389 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:09.446638 24015 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:09.458635 25922 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.461019 25912 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:09.458793 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.465548 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:09.466325 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49487.7us
I20260709 12:14:09.461019 25922 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.470913 23461 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:09.471992 25924 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.473294 25848 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:09.473809 25848 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.476504 25848 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.478394 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.479781 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.480104 25899 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.475457 23320 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:14:09.482321 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.483556 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.483884 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.483593 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.485121 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.485318 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.483657 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:09.491192 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.416934 25702 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.748s	user 0.020s	sys 0.005s
I20260709 12:14:09.493939 25913 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.495182 23314 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:09.495291 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.495946 24007 raft_consensus.cc:2376] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
I20260709 12:14:09.496206 25913 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.465574 24013 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:09.498032 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.499015 24015 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:09.498890 23392 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:09.502583 25922 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.511163 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.516050 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:09.520093 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.513041 25922 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.547394 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.547607 25943 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:09.548250 24016 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:09.549646 26058 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:09.550573 25950 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:09.550974 25950 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:09.551328 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.551954 25950 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.552552 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.553169 25950 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.550985 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.554420 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.557391 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.555109 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.558249 24013 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:09.521700 25924 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.557454 25702 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:14:09.561612 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.561748 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.562786 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.562969 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.562727 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.565521 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.566223 23466 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:09.568251 24016 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:09.568671 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.569494 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.570380 23462 raft_consensus.cc:2410] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:09.570996 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.575587 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:09.570672 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:09.578255 23466 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:09.579103 23392 raft_consensus.cc:1217] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 12:14:09.580029 24014 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
W20260709 12:14:09.584223 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1520494809999491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 12:14:09.585796 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.613780 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.587091 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:09.586695 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.587311 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.594779 25922 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.595709 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.596701 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.620569 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.599751 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.604851 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.605801 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.614043 23392 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:14:09.618692 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74211.2us
I20260709 12:14:09.617792 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:09.619457 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1493494809999447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260709 12:14:09.576726 25943 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:14:09.620676 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:09.598914 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 750) took 1010 ms. Trace:
I20260709 12:14:09.618264 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.602928 25899 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.634987 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.579728 25993 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.641060 23465 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:09.637122 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.644032 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.644528 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.645459 26015 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:09.647591 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.590260 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.555186 26060 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:14:09.625049 23391 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9. Re-sending same reply.
I20260709 12:14:09.653071 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.653553 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.653695 23462 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:14:09.625875 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0098494809999465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 12:14:09.637871 23390 raft_consensus.cc:2368] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:14:09.649775 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.655254 24013 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:09.654263 23464 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:09.656297 25775 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:09.656745 25775 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.657178 25775 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:09.658306 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.658601 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.659289 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.659817 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.660375 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.660908 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.662525 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.658329 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.663295 23396 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:09.666011 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 79022.7us
I20260709 12:14:09.650940 24016 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:09.673934 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.674638 23391 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:09.679737 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.682979 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.683704 24011 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:09.675150 25848 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.634099 24010 rpcz_store.cc:276] 0709 12:14:08.588597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.592836 (+  4239us) service_pool.cc:224] Handling call
0709 12:14:08.665948 (+ 73112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.306848 (+640900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.598546 (+291698us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54006bdad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:09.598826 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469203328}
I20260709 12:14:09.687454 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:09.687753 23219 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 217) took 16961 ms (client timeout 15000 ms). Trace:
W20260709 12:14:09.688114 23219 rpcz_store.cc:269] 0709 12:13:52.657689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.657848 (+   159us) service_pool.cc:224] Handling call
0709 12:13:54.157200 (+1499352us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:09.614835 (+15457635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:03.359344 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:03.460034 (+100690us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:03.460097 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:14:03.460115 (+    18us) tablet.cc:662] Decoding operations
0709 12:14:03.595723 (+135608us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:03.607922 (+ 12199us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:03.609982 (+  2060us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:14:03.686729 (+ 76747us) tablet.cc:701] Row locks acquired
0709 12:14:03.686777 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:14:03.686951 (+   174us) write_op.cc:270] Start()
0709 12:14:03.687109 (+   158us) write_op.cc:276] Timestamp: P: 1783599243686887 usec, L: 0
0709 12:14:03.687131 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:14:03.714066 (+ 26935us) log.cc:844] Serialized 2084 byte log entry
0709 12:14:03.929833 (+215767us) op_driver.cc:464] REPLICATION: finished
0709 12:14:04.167123 (+237290us) write_op.cc:301] APPLY: starting
0709 12:14:04.169255 (+  2132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:04.172083 (+  2828us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:04.182616 (+ 10533us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:04.230256 (+ 47640us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:04.230584 (+   328us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:14:04.230665 (+    81us) write_op.cc:312] APPLY: finished
0709 12:14:04.230999 (+   334us) log.cc:844] Serialized 47 byte log entry
0709 12:14:04.231233 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:04.231355 (+   122us) write_op.cc:454] Released schema lock
0709 12:14:04.231414 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:04.241444 (+ 10030us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:04.247258 (+  5814us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":336705,"reported_tablets":21,"spinlock_wait_cycles":48280448,"child_traces":[["op",{"apply.queue_time_us":216609,"apply.run_cpu_time_us":3306,"apply.run_wall_time_us":79345,"num_ops":5,"prepare.queue_time_us":776588,"prepare.run_cpu_time_us":37910,"prepare.run_wall_time_us":463903,"raft.queue_time_us":97092,"raft.run_cpu_time_us":29638,"raft.run_wall_time_us":177511,"replication_time_us":242468,"spinlock_wait_cycles":49024,"thread_start_us":410535,"threads_started":4,"wal-append.queue_time_us":95907,"wal-append.run_cpu_time_us":19564,"wal-append.run_wall_time_us":1423458}]]}
I20260709 12:14:09.693388 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.662421 25870 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:14:09.698104 25980 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:09.699092 23315 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:09.651962 26015 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:09.702260 26015 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.703858 25993 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.686924 25848 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.702210 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:09.715701 23362 leader_election.cc:400] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6.
I20260709 12:14:09.718343 23362 leader_election.cc:403] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:09.716419 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:09.735926 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:09.737195 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:09.741773 26054 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.404s	user 0.000s	sys 0.003s
W20260709 12:14:09.743196 26055 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:14:09.737864 23390 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:09.748423 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:09.747037 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.751967 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.753834 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.754573 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.754647 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.755672 23462 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.761380 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:09.748447 26070 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:14:09.736307 25993 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.745847 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:09.769698 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.770275 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.770987 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.771354 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:09.772830 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:09.766479 26060 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.784789 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:09.789264 23391 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:09.766893 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:09.790941 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:09.795547 23318 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:09.798332 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:09.765336 26015 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:09.807112 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:09.805753 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 151127us
I20260709 12:14:09.808555 23396 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:09.811780 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:09.813499 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46688.9us
I20260709 12:14:09.815411 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:09.795408 25660 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:09.819077 23464 raft_consensus.cc:2410] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:09.796629 23461 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:09.813014 25884 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:09.857067 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.791726 25899 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:14:09.796151 25937 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:14:09.808604 25862 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:14:09.799304 25993 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.820399 25660 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.878932 25660 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:09.818646 25661 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:14:09.864183 25702 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
W20260709 12:14:09.867492 26063 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:14:09.869596 25993 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:09.817200 25973 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:14:09.874524 26072 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:09.887786 26013 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:09.888234 26013 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:09.878398 26068 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:09.890383 24009 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:14:09.891037 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 756) took 1052 ms. Trace:
I20260709 12:14:09.891376 24009 rpcz_store.cc:276] 0709 12:14:08.838926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.843493 (+  4567us) service_pool.cc:224] Handling call
0709 12:14:09.889830 (+1046337us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:09.890023 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.890762 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.890978 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6674176}
W20260709 12:14:09.893267 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47304.3us
I20260709 12:14:09.888187 26072 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.853829 26015 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:09.893954 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:14:09.895892 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.910156 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.896692 26072 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.908517 26059 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:09.907004 26027 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:09.911933 26015 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.900934 25739 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 12:14:09.919797 26053 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.582s	user 0.002s	sys 0.001s
I20260709 12:14:09.915949 26027 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.896281 25882 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 12:14:09.922017 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.916276 26011 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:09.922748 26011 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.923787 25681 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:09.924172 25681 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.924790 26011 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.915735 26059 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:09.925545 23287 connection.cc:256] RPC call timeout handler was delayed by -1.7055791849998059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 12:14:09.925819 24015 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:09.925944 23287 connection.cc:256] RPC call timeout handler was delayed by -1.6506791849999445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 12:14:09.926244 26059 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.927645 26059 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:09.928124 26060 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6.
I20260709 12:14:09.921502 26027 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.904027 25775 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.929889 26027 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:09.926262 23287 connection.cc:256] RPC call timeout handler was delayed by -1.6479791849999401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:14:09.931401 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5399791849997655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
I20260709 12:14:09.931561 25775 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:09.933072 26023 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.934212 25884 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:09.889344 26013 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:09.940140 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:09.942538 23434 leader_election.cc:336] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.502s (SENT)
I20260709 12:14:09.943125 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:09.938076 26013 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:09.944015 26011 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:14:09.915508 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60912.2us
I20260709 12:14:09.946254 23462 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:09.931803 23287 connection.cc:256] RPC call timeout handler was delayed by -1.3176791849998608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 12:14:09.947369 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1439791849998073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
I20260709 12:14:09.925050 25681 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.950246 25681 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:09.952358 26010 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:09.952790 26010 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.953589 26010 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:09.955063 26010 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:09.955121 24008 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:09.966771 25948 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:09.967366 25948 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:09.967537 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:09.968691 26009 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:14:09.968518 25948 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:09.969236 26009 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:09.969875 26009 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:14:09.987986 23434 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.566s (SENT)
I20260709 12:14:09.990703 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:09.991423 23388 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:09.992846 25948 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:09.998129 23321 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:09.998593 23217 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 6, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.000147 26027 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:10.001322 25702 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:10.001739 25702 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:10.000958 23217 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 6, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.002569 25702 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:10.003726 25702 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:10.003118 23217 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.005142 23217 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 6, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.006891 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:10.010152 23321 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:10.014305 25884 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:14:10.021610 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 54161.3us
W20260709 12:14:10.024258 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 212404us
I20260709 12:14:10.027683 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.044893 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:10.050032 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.090234 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:10.098505 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 261116us
W20260709 12:14:10.089738 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 350795us
I20260709 12:14:10.103114 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.105001 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.010586 23217 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194), now has a pending config: VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193), VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194), VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195), VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:10.123916 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:10.126859 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.705s (SENT)
I20260709 12:14:10.127355 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:10.134101 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 89979.3us
I20260709 12:14:10.134506 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:10.129498 26024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.003s	sys 0.015s
I20260709 12:14:10.125371 26011 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:10.104741 25660 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:10.138011 25950 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:10.143476 26024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.003s	sys 0.015s
I20260709 12:14:10.151206 23217 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 6, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.135537 26079 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:10.167217 23217 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:10.169747 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68878.3us
I20260709 12:14:10.158691 26079 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.162351 25922 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:10.180771 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.177938 26071 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:10.186226 26076 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:10.193876 26071 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.194164 26076 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.194423 26071 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.194670 25660 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:10.195808 26009 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:10.199589 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.203361 23388 raft_consensus.cc:2368] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9.
I20260709 12:14:10.201625 23217 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.194571 26076 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:10.199996 25922 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:10.205098 23395 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:10.205613 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:10.218639 23313 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:10.180727 26079 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:10.207266 26059 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.208068 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:10.161861 26023 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:14:10.228699 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.229597 23396 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:10.235944 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:10.236131 23394 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:10.238445 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:10.239032 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66949.6us
I20260709 12:14:10.254040 23217 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.261325 23217 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.263463 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.264782 23396 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:10.266639 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:10.208619 26072 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:10.240062 26011 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:10.285399 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.291493 23321 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:10.293926 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.268757 25950 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:10.230914 26059 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:10.298427 23217 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 1 to 2, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.310081 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.312382 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:10.326781 23394 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:14:10.338986 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 99595.5us
W20260709 12:14:10.349188 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.300638 26009 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:10.349624 23217 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.317163 26072 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:10.300700 26098 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:10.352738 23217 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:10.357573 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:10.351044 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:10.362308 23217 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:10.369861 23217 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 4, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:10.373253 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 246404us
I20260709 12:14:10.373618 23394 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:10.374423 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:10.377151 23395 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:10.376619 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:10.379230 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:10.386592 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.502s (SENT)
I20260709 12:14:10.387203 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:10.405233 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 302194us
W20260709 12:14:10.407796 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 270721us
I20260709 12:14:10.326371 25922 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:10.420591 23434 leader_election.cc:336] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.734s (SENT)
I20260709 12:14:10.421088 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:10.425498 26103 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:10.426344 26103 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:10.429843 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:10.431556 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:10.432310 24016 raft_consensus.cc:2376] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6. Re-sending same reply.
W20260709 12:14:10.438719 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.443661 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.666s (SENT)
I20260709 12:14:10.444211 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.445852 26106 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:10.445989 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.446499 26106 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:10.447549 26106 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:10.449731 26099 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:10.450330 26099 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:10.450836 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:10.451319 26099 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:10.453771 23319 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:10.453563 26011 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:10.475732 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:10.483753 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:10.484880 24016 raft_consensus.cc:2410] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:10.484440 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:10.468501 26009 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:10.506811 23466 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:10.494451 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 131560us
I20260709 12:14:10.510299 23312 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:10.477187 26103 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.520082 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:10.402112 25950 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:10.523171 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:10.526389 23320 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.528580 23321 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:10.527139 24016 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:10.529364 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:10.531414 23469 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:10.528937 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:10.513952 26009 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:10.533849 23463 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:10.540469 23321 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:10.543689 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:10.544271 24011 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:10.544406 23434 leader_election.cc:336] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.654s (SENT)
I20260709 12:14:10.546293 24007 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.546617 23464 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:10.516002 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:10.552695 24011 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:10.572887 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:10.574316 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:10.580775 23313 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:10.551738 26009 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:10.553890 26059 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:10.563534 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:10.601576 23319 raft_consensus.cc:606] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:10.603269 23319 raft_consensus.cc:614] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:10.606185 26015 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:10.525055 26103 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:10.563743 26109 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:10.613653 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.622916 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:10.623600 23464 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:10.562510 25739 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:14:10.595608 24008 raft_consensus.cc:2368] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9.
I20260709 12:14:10.565016 26105 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:10.636312 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.043s (SENT)
I20260709 12:14:10.640542 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:10.641758 26105 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:10.615264 24012 raft_consensus.cc:2376] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7. Re-sending same reply.
W20260709 12:14:10.931717 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 399924us
W20260709 12:14:10.961556 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 438953us
W20260709 12:14:10.970937 26011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.018s	sys 0.011s
W20260709 12:14:11.061718 23316 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.016s	sys 0.016s
W20260709 12:14:11.081199 26071 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.014s
I20260709 12:14:11.063541 23466 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:11.068792 23982 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.006s	sys 0.017s
W20260709 12:14:11.070758 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 434457us
W20260709 12:14:11.107316 23434 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.739s (SENT)
W20260709 12:14:11.108321 23982 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.006s	sys 0.017s
W20260709 12:14:11.108827 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 663945us
W20260709 12:14:11.071187 26073 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.007s	sys 0.008s
I20260709 12:14:11.072417 26110 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:11.073096 25681 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:14:11.073662 23388 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.835s	user 0.013s	sys 0.008s
I20260709 12:14:11.187355 23388 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:11.074405 26059 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:10.674901 25993 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.147s
W20260709 12:14:11.061277 23470 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.000s	sys 0.031s
I20260709 12:14:11.077582 23317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
W20260709 12:14:11.077785 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.078562 26079 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.842s	user 0.009s	sys 0.009s
W20260709 12:14:11.079062 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 559060us
I20260709 12:14:11.081781 26117 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.083753 26103 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:11.084857 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.586900 26080 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.069096 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
W20260709 12:14:11.093784 23316 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.016s	sys 0.017s
W20260709 12:14:11.036835 23396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.005s	sys 0.009s
I20260709 12:14:11.094823 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:11.102543 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:11.110562 23469 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.005s
I20260709 12:14:11.131774 26071 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:11.152315 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45044.5us
W20260709 12:14:11.102329 26011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.018s	sys 0.011s
W20260709 12:14:11.373057 23470 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.000s	sys 0.045s
I20260709 12:14:11.373972 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 430) took 1156 ms. Trace:
I20260709 12:14:11.374435 23388 rpcz_store.cc:276] 0709 12:14:10.217319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.218314 (+   995us) service_pool.cc:224] Handling call
0709 12:14:11.373894 (+1155580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:11.375655 23317 raft_consensus.cc:606] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:11.376089 23317 raft_consensus.cc:614] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:11.383210 23469 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:11.399441 23396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.008s	sys 0.013s
I20260709 12:14:11.271550 23319 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
W20260709 12:14:11.407510 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 305041us
W20260709 12:14:11.408950 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 301349us
W20260709 12:14:11.121694 26073 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.007s	sys 0.008s
W20260709 12:14:11.410228 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.186868 26072 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.248068 26119 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:11.439903 23317 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:11.252823 26142 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:11.106320 26105 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:11.454283 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 359484us
W20260709 12:14:11.474679 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61652.8us
I20260709 12:14:11.382546 26071 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.415729 26072 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.486230 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:11.439164 26109 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.024s	sys 0.033s
I20260709 12:14:11.380324 26079 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.260506 26133 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:11.493125 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83187.1us
I20260709 12:14:11.193296 26110 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:11.509025 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67943.4us
I20260709 12:14:11.514816 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:11.511204 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:11.520536 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:11.521298 23464 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:11.521941 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44870.4us
W20260709 12:14:11.501302 26109 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.025s	sys 0.033s
I20260709 12:14:11.431206 26080 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:11.527117 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50444.1us
I20260709 12:14:11.480089 26125 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:11.525851 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.501194 25660 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:11.531747 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:11.534705 23317 raft_consensus.cc:606] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:11.524296 26076 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.294s	user 0.010s	sys 0.010s
I20260709 12:14:11.570865 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 570) took 1261 ms. Trace:
I20260709 12:14:11.508764 26010 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.580812 23317 raft_consensus.cc:614] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:11.581914 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:11.522850 26023 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.526698 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1775) took 1095 ms. Trace:
I20260709 12:14:11.525635 25993 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.608640 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 99407.6us
I20260709 12:14:11.546115 26139 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:11.591130 23316 rpcz_store.cc:276] 0709 12:14:10.273867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.274028 (+   161us) service_pool.cc:224] Handling call
0709 12:14:10.298249 (+ 24221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.298476 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:10.465355 (+166879us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.534484 (+1069129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:11.534540 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:11.534588 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.534673 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.534806 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12448,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":32,"spinlock_wait_cycles":61612928,"thread_start_us":1211921,"threads_started":3,"wal-append.queue_time_us":608537}
I20260709 12:14:11.547096 26109 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:11.620464 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 241392us
W20260709 12:14:11.629151 23434 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.169s (SENT)
W20260709 12:14:11.630159 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.823s (SENT)
I20260709 12:14:11.630654 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:11.529390 26127 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.461557 26117 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:11.585069 23464 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:11.588747 25660 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.485468 26142 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:11.641111 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 118406us
I20260709 12:14:11.642525 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:11.643575 25950 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:11.644582 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:11.629618 26009 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.990s	user 0.014s	sys 0.002s
I20260709 12:14:11.648176 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:11.621614 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:11.627805 26139 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:11.588438 26118 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:14:11.638805 25660 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.636291 26133 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:11.615763 26010 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.654083 25950 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:11.566709 26119 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.596639 26076 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.789330 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:11.622731 26023 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.694079 26139 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:11.805335 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 154936us
I20260709 12:14:11.634356 25993 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:11.821270 26136 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
W20260709 12:14:11.822067 26130 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
W20260709 12:14:11.822446 26140 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
W20260709 12:14:11.824421 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48783.7us
I20260709 12:14:11.614936 26125 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:11.819679 26158 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:11.825562 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:11.659469 26009 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:11.784336 25950 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.805919 26110 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.792096 26099 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:11.830410 24011 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:11.829986 26072 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.910584 26153 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:11.911043 26072 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:11.911113 26153 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.911262 26080 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.911723 26153 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:11.911908 26080 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.911664 26072 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.912470 26080 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:11.913025 23495 heartbeater.cc:646] Failed to heartbeat to 127.21.150.254:44861 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.150.254:44861 timed out after 15.000s (SENT)
W20260709 12:14:11.839555 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.850015 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.639925 26127 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:11.915030 26111 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:14:11.916384 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1776) took 1500 ms. Trace:
I20260709 12:14:11.916729 26127 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:11.914170 26143 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260709 12:14:11.917505 26123 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:14:11.917690 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 770) took 1374 ms. Trace:
I20260709 12:14:11.918155 24007 rpcz_store.cc:276] 0709 12:14:10.543644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.547175 (+  3531us) service_pool.cc:224] Handling call
0709 12:14:10.547517 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.587580 (+ 40063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.593954 (+  6374us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.675213 (+ 81259us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.136634 (+461421us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:11.138246 (+  1612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.139821 (+  1575us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.917404 (+777583us) raft_consensus.cc:1685] finished
0709 12:14:11.917444 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.917605 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14500,"raft.queue_time_us":38324,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":68018,"threads_started":1,"wal-append.queue_time_us":457670}
I20260709 12:14:11.919656 26142 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.920650 26023 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:11.921757 26125 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.922073 26072 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.922168 25660 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:11.923098 26072 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:11.924192 26076 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.916746 23462 rpcz_store.cc:276] 0709 12:14:10.415852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.415999 (+   147us) service_pool.cc:224] Handling call
0709 12:14:10.416436 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.416744 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.416785 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.422719 (+  5934us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.770768 (+348049us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:10.770823 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.770927 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.916142 (+1145215us) raft_consensus.cc:1685] finished
0709 12:14:11.916166 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.916321 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":347606,"threads_started":1,"wal-append.queue_time_us":1396752}
I20260709 12:14:11.931398 23463 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:11.905260 23287 leader_election.cc:400] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:14:11.940778 23287 leader_election.cc:403] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:11.909037 25950 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:11.951715 26009 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:11.951108 26125 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:11.967412 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 575) took 1358 ms. Trace:
W20260709 12:14:11.965833 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 451098us
I20260709 12:14:11.955884 26110 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:11.970266 23320 rpcz_store.cc:276] 0709 12:14:10.608679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.610365 (+  1686us) service_pool.cc:224] Handling call
0709 12:14:10.637551 (+ 27186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.638023 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:10.638073 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.638634 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.014859 (+376225us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:11.014912 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.015015 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.967179 (+952164us) raft_consensus.cc:1685] finished
0709 12:14:11.967197 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.967345 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":434191,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":375638,"threads_started":1,"wal-append.queue_time_us":841205}
W20260709 12:14:11.977072 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 325164us
I20260709 12:14:11.953126 26142 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:14:11.962239 26079 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.964350 26162 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.965118 24009 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.003s	sys 0.004s
I20260709 12:14:11.974313 25660 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:12.010497 24009 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.004s	sys 0.004s
W20260709 12:14:12.035778 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 179132us
I20260709 12:14:12.043839 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:11.974684 26125 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:12.071583 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83929.3us
I20260709 12:14:11.851743 26109 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.019412 26119 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:12.065554 23434 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.525s (SENT)
W20260709 12:14:12.087723 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44175.4us
I20260709 12:14:12.088987 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:12.094529 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.095381 23315 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:12.025808 26151 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.035362 26009 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:12.211268 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 122291us
W20260709 12:14:12.213480 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 124548us
W20260709 12:14:12.153399 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74512.6us
I20260709 12:14:12.214246 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:12.165709 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 100196us
I20260709 12:14:12.041106 26010 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.216389 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:12.215212 23434 leader_election.cc:336] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.904s (SENT)
I20260709 12:14:12.217252 23318 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:12.217732 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:12.041677 26158 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:12.226222 23981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.007s	sys 0.000s
W20260709 12:14:12.226625 23981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.008s	sys 0.000s
I20260709 12:14:12.073999 26117 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:12.228317 26117 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:12.228706 26117 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.229040 26117 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:12.232545 26009 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.233597 26165 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:12.234167 26165 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.235332 26009 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.236279 26165 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:12.012015 26162 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.238965 26162 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.239545 26162 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:12.245613 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 420123us
I20260709 12:14:12.246589 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.249805 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.250402 23315 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.250687 26125 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:11.614157 23461 rpcz_store.cc:276] 0709 12:14:10.413534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.414623 (+  1089us) service_pool.cc:224] Handling call
0709 12:14:10.439174 (+ 24551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.439444 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:10.974269 (+534825us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.439583 (+465314us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:11.454534 (+ 14951us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:11.475293 (+ 20759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.487794 (+ 12501us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.509085 (+ 21291us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":168288,"prepare.queue_time_us":635690,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":186051,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":30105,"thread_start_us":575088,"threads_started":2,"wal-append.queue_time_us":732769}
W20260709 12:14:12.263893 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48698.1us
W20260709 12:14:12.264909 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.941s (SENT)
W20260709 12:14:12.209362 26122 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.458s	user 0.000s	sys 0.001s
I20260709 12:14:12.265383 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.285508 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.312314 23318 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:12.286319 25950 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:12.290516 26154 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:14:12.313798 23315 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:12.292570 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 771) took 1748 ms. Trace:
I20260709 12:14:12.294564 26137 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:12.295372 26108 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 12:14:12.305716 26170 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:12.244233 26139 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.306951 26080 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.309324 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1782) took 1797 ms. Trace:
I20260709 12:14:12.353906 23466 rpcz_store.cc:276] 0709 12:14:10.512280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.522237 (+  9957us) service_pool.cc:224] Handling call
0709 12:14:10.522548 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.119725 (+597177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.308996 (+1189271us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54001068d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:12.309242 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229436288}
W20260709 12:14:12.312584 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47721.5us
I20260709 12:14:12.314184 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.355944 23313 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:14:12.355989 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.955s (SENT)
I20260709 12:14:12.213073 26119 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:12.359195 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 112726us
I20260709 12:14:12.315511 24011 rpcz_store.cc:276] 0709 12:14:10.543916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.565716 (+ 21800us) service_pool.cc:224] Handling call
0709 12:14:10.566042 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:11.909683 (+1343641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.292236 (+382553us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540044ff54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:12.292497 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2907264896}
I20260709 12:14:12.316639 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.217242 26156 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.259s
I20260709 12:14:12.340276 23315 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:12.356577 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.241325 26152 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.361889 23321 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:12.362964 23314 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:12.370025 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.370924 23318 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:12.372889 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.378587 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.379251 23313 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:12.379951 26152 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.380457 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1779) took 1912 ms. Trace:
I20260709 12:14:12.380862 23470 rpcz_store.cc:276] 0709 12:14:10.468296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.469113 (+   817us) service_pool.cc:224] Handling call
0709 12:14:10.483830 (+ 14717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.519717 (+ 35887us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:11.373649 (+853932us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.604900 (+231251us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:11.605484 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.605645 (+   161us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:11.605685 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.605774 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.380257 (+774483us) raft_consensus.cc:1685] finished
0709 12:14:12.380275 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.380399 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11169,"prepare.queue_time_us":555742,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":50724,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":25092,"thread_start_us":990392,"threads_started":2,"wal-append.queue_time_us":601879}
I20260709 12:14:12.382470 26152 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:12.383819 26135 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:12.384352 26135 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.386441 26135 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:12.386950 26135 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:12.387705 23313 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:12.387513 26158 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.388178 26177 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:12.388994 26177 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:12.389842 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.341552 26170 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:12.387722 26135 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:12.391268 26170 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:12.394322 25993 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.395897 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 441) took 1992 ms. Trace:
I20260709 12:14:12.396260 23396 rpcz_store.cc:276] 0709 12:14:10.403492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.403674 (+   182us) service_pool.cc:224] Handling call
0709 12:14:10.439891 (+ 36217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.440307 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.440351 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.458159 (+ 17808us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.399980 (+941821us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:11.400017 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.400090 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.395668 (+995578us) raft_consensus.cc:1685] finished
0709 12:14:12.395688 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.395818 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8952,"raft.queue_time_us":21736,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6528,"thread_start_us":907803,"threads_started":1,"wal-append.queue_time_us":890573}
I20260709 12:14:12.397688 26135 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:12.398227 26135 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:12.403654 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.311524 26009 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:14:12.410173 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.040s	sys 0.060s
I20260709 12:14:12.413523 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.414566 23313 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:12.414794 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.040s	sys 0.060s
I20260709 12:14:12.239957 26162 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:12.415318 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 833513us
W20260709 12:14:12.415858 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.441686 23315 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:12.443264 23315 raft_consensus.cc:606] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:12.443773 23315 raft_consensus.cc:614] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:12.411494 26119 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:14:12.452458 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 79674.7us
I20260709 12:14:12.453608 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.457659 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:12.458977 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.414458 26157 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:12.461993 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.466727 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.462997 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.347656 26137 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.468693 23314 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
W20260709 12:14:12.481171 26124 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.407s	user 0.004s	sys 0.007s
I20260709 12:14:12.451707 26119 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:12.482128 26119 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.485447 23314 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:54652
I20260709 12:14:12.487484 26139 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.488258 26162 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.494781 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:12.496515 23321 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:12.467011 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:12.500428 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.506752 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.507617 23313 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
W20260709 12:14:12.485188 26105 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.990s	user 0.006s	sys 0.003s
I20260709 12:14:12.512799 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:12.449218 25950 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.520493 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.521257 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:12.522859 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 56269.6us
I20260709 12:14:12.525509 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:12.526643 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 313677us
I20260709 12:14:12.528872 23315 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:12.515650 26156 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.512274 26009 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:12.535565 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.539520 23313 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:12.538342 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.542080 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:12.543048 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.035s	sys 0.064s
W20260709 12:14:12.543449 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.036s	sys 0.064s
I20260709 12:14:12.543514 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:12.510331 26165 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:14:12.547063 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 191110us
I20260709 12:14:12.550382 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:12.551555 24011 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:12.497462 26157 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.555761 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.556054 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:12.556555 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:12.556931 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:12.557186 23468 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:12.557600 24013 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:12.559896 23318 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:12.556470 23313 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:12.570853 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:12.571695 24012 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:12.572045 26161 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:12.573652 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 762) took 2060 ms. Trace:
I20260709 12:14:12.516880 26105 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.574020 24015 rpcz_store.cc:276] 0709 12:14:10.513599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.519150 (+  5551us) service_pool.cc:224] Handling call
0709 12:14:10.529594 (+ 10444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.590049 (+ 60455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.590102 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.602478 (+ 12376us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.102161 (+499683us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:11.502754 (+400593us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.614481 (+111727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.614578 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.573406 (+958828us) raft_consensus.cc:1685] finished
0709 12:14:12.573453 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.573590 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74580,"prepare.queue_time_us":532596,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":8090,"raft.queue_time_us":322177,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"spinlock_wait_cycles":15327616,"thread_start_us":780036,"threads_started":3,"wal-append.queue_time_us":1875563}
I20260709 12:14:12.586285 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:12.587169 24014 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:12.572894 26161 consensus_queue.cc:1059] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:12.589185 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:12.590039 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.590792 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:12.594285 23434 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.692s (SENT)
I20260709 12:14:12.594952 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.594487 23467 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:12.597534 23464 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:12.598054 23313 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:12.598893 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 85812.6us
W20260709 12:14:12.601307 23434 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.728s (SENT)
I20260709 12:14:12.599541 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:12.601898 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:12.603344 23465 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:12.602643 26174 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.608760 24008 raft_consensus.cc:2410] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:14:12.609323 26180 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:14:12.609825 26167 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.610332 26167 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:12.612725 26141 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.528s	user 0.002s	sys 0.009s
I20260709 12:14:12.612896 26173 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:12.613564 26173 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:12.614152 26113 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 1.541s	user 0.007s	sys 0.007s
I20260709 12:14:12.614595 26173 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.616235 24013 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:12.615252 26112 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.543s	user 0.000s	sys 0.022s
I20260709 12:14:12.573519 26171 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:12.581763 26080 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.609236 26163 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:12.608645 26172 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:12.621251 23434 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.080s (SENT)
W20260709 12:14:12.628458 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 115730us
I20260709 12:14:12.656695 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.625860 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.658094 23313 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:12.601747 26153 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.000s
I20260709 12:14:12.602299 26164 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:12.655450 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:12.603929 26011 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:12.663805 23314 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.617666 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:12.612722 26167 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:12.629575 26177 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:12.629168 26009 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:12.589963 26170 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.680778 26179 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:12.680930 26184 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:12.664125 26153 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.680650 26175 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.655761 26080 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.662318 26164 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.632414 26151 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5.
I20260709 12:14:12.664292 26172 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.680778 26189 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:14:12.682586 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61358.3us
I20260709 12:14:12.683068 26137 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:12.683585 26146 raft_consensus.cc:2764] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:12.683415 26117 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.685276 26157 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:12.697137 26179 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:12.701416 23434 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.105s (SENT)
W20260709 12:14:12.722380 23434 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.146s (SENT)
I20260709 12:14:12.680007 26177 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.702611 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.704728 23318 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:12.705720 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:12.709010 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 51373us
I20260709 12:14:12.735852 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.721275 23312 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:12.724730 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 476) took 1211 ms. Trace:
I20260709 12:14:12.727810 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:12.738030 23434 leader_election.cc:336] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.158s (SENT)
I20260709 12:14:12.739476 23313 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:12.608657 26174 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.699062 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:12.716097 26106 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.247s	user 0.009s	sys 0.047s
I20260709 12:14:12.746925 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:12.739542 23469 rpcz_store.cc:276] 0709 12:14:11.513272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.514147 (+   875us) service_pool.cc:224] Handling call
0709 12:14:11.529865 (+ 15718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.530080 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:11.595209 (+ 65129us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.255118 (+659909us) spinlock_profiling.cc:243] Waited 9.02 ms on lock 0x7b18001f8520. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:14:12.724018 (+468900us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.724261 (+   243us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:12.724327 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:12.724394 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.724486 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.724641 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16710,"prepare.queue_time_us":63511,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":7162,"raft.queue_time_us":10009,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7630,"spinlock_wait_cycles":20737408,"thread_start_us":468139,"threads_started":2,"wal-append.queue_time_us":1099822}
I20260709 12:14:12.700971 26189 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:12.742861 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:12.744607 23317 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7.
I20260709 12:14:12.700871 26164 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:12.741619 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 79057.3us
W20260709 12:14:12.750470 23434 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.174s (SENT)
I20260709 12:14:12.750466 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:12.771310 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:12.782521 23434 leader_election.cc:336] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.800s (SENT)
I20260709 12:14:12.783071 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.731760 26139 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:12.731216 26179 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:12.770061 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:12.803848 23391 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:12.736743 26177 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:12.719347 26165 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:14:12.811905 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:12.813413 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:12.746523 26106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.290s	user 0.010s	sys 0.047s
I20260709 12:14:12.815871 23319 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:14:12.822700 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:12.822868 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:12.823654 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:12.741200 26157 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:14:12.719002 26135 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:12.818167 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:12.817034 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 77912.3us
I20260709 12:14:12.828810 23393 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:12.797204 23394 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:12.823145 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:12.733212 26190 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:14:12.700698 26175 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:12.848222 23436 leader_election.cc:336] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 2.239s (SENT)
I20260709 12:14:12.773164 24007 raft_consensus.cc:2376] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7. Re-sending same reply.
I20260709 12:14:12.904934 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:12.904506 23315 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:12.904184 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 121689us
I20260709 12:14:12.902652 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:12.845319 23387 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:12.904176 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.921659 23319 raft_consensus.cc:2410] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:14:12.923146 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.086s (SENT)
I20260709 12:14:12.926868 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:12.947129 23434 leader_election.cc:336] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.173s (SENT)
I20260709 12:14:12.947566 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:12.948671 23434 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 3.324s (SENT)
I20260709 12:14:12.949059 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:12.900566 26185 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:14:12.822471 26011 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:12.829706 26167 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:12.860818 26169 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:12.899649 26153 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:12.969692 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 121508us
I20260709 12:14:12.974632 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.979770 23319 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:12.977551 26009 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:12.982419 26135 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.983839 26175 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:12.951988 26184 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:12.989605 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 231870us
I20260709 12:14:12.946241 26105 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:12.953397 26106 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:12.953701 26137 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:12.988521 26184 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:12.826457 26170 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:13.023154 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.023522 23315 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:12.979532 26156 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:12.977766 26163 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:13.028412 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.980075 26169 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.030848 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:12.971676 26167 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.034106 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.070159 23315 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:13.036055 23319 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:13.038230 26172 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:13.020666 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 201887us
W20260709 12:14:13.046718 26191 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:14:13.073302 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:13.048111 26174 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.048681 26128 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:13.050050 26177 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:13.065979 26183 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
W20260709 12:14:13.066979 26202 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:14:13.068202 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 230274us
I20260709 12:14:13.092782 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:13.069927 26169 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:13.102782 23394 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:13.103389 23396 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:13.101392 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:13.076357 26167 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:13.092348 26186 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.394s	user 0.000s	sys 0.001s
I20260709 12:14:13.129513 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.141048 26177 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.141928 26184 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:13.144196 26200 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:13.144770 26200 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:13.146579 26165 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:13.149784 26200 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.151968 26200 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:13.153056 26188 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.320s	user 0.000s	sys 0.000s
I20260709 12:14:13.099587 26128 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:13.103104 26119 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.002s
W20260709 12:14:13.176246 26134 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 2.101s	user 0.002s	sys 0.018s
I20260709 12:14:13.185206 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 764) took 2658 ms. Trace:
I20260709 12:14:13.185819 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:13.189636 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1786) took 2653 ms. Trace:
I20260709 12:14:13.190187 23463 rpcz_store.cc:276] 0709 12:14:10.536163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.560918 (+ 24755us) service_pool.cc:224] Handling call
0709 12:14:10.577119 (+ 16201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.950067 (+1372948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.189318 (+1239251us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:13.189556 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3101066112}
I20260709 12:14:13.107146 26080 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.185724 24009 rpcz_store.cc:276] 0709 12:14:10.526741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.528651 (+  1910us) service_pool.cc:224] Handling call
0709 12:14:10.541286 (+ 12635us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:10.564791 (+ 23505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.572539 (+  7748us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:10.637010 (+ 64471us) log.cc:844] Serialized 52 byte log entry
0709 12:14:11.079966 (+442956us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:12.047206 (+967240us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.321683 (+274477us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.390469 (+ 68786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.390542 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:13.128608 (+738066us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:13.184941 (+ 56333us) raft_consensus.cc:1685] finished
0709 12:14:13.184966 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.185129 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93284,"prepare.queue_time_us":693613,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":83,"raft.queue_time_us":122465,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":105827,"spinlock_wait_cycles":147853056,"thread_start_us":1228671,"threads_started":2,"wal-append.queue_time_us":437679}
W20260709 12:14:13.197710 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.123668 26009 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:12.962862 26189 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.104559 26172 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:14:13.104117 26137 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:14:13.225015 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.231251 23319 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:13.199025 26098 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.246883 23434 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.843s (SENT)
I20260709 12:14:13.196518 26184 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:14:13.222553 26080 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.181352 26119 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.196926 25950 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:14:13.262511 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.226294 26009 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:13.268694 23319 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:14:13.254097 26157 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:13.255146 26189 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:13.275367 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.275947 26192 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:13.281164 26197 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 12:14:13.286893 26131 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 2.212s	user 0.011s	sys 0.016s
I20260709 12:14:13.287840 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:13.291958 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 185111us
I20260709 12:14:13.299667 23319 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.264226 26079 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:13.304781 23321 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.307125 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:13.312808 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:13.314430 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.315294 23321 raft_consensus.cc:2368] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4.
I20260709 12:14:13.273738 26157 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:13.292695 26192 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.260936 26098 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:13.330626 23981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.024s
I20260709 12:14:13.332839 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.333539 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.268057 26080 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.334131 23319 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:13.334143 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.339526 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.339816 23321 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:13.340752 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:13.336403 23981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.014s	sys 0.024s
I20260709 12:14:13.341374 23319 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:13.298840 25994 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:14:13.341912 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.340770 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.345716 23321 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.344177 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.343848 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:13.323979 26211 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:13.330433 25920 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:13.274324 26164 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.359910 26027 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:13.357757 26211 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.363862 26217 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:13.438802 26217 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.364355 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:13.424472 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 767) took 2881 ms. Trace:
I20260709 12:14:13.335000 26192 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:13.429858 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.372705 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.449940 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:13.317713 26079 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.452208 23470 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:13.453557 23982 connection.cc:256] RPC call timeout handler was delayed by -1.3969422850000228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260709 12:14:13.456331 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.456950 24011 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:13.443204 24010 rpcz_store.cc:276] 0709 12:14:10.542744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.545011 (+  2267us) service_pool.cc:224] Handling call
0709 12:14:10.545342 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.585787 (+ 40445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.585820 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:13.105771 (+2519951us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:13.109131 (+  3360us) log.cc:844] Serialized 26 byte log entry
0709 12:14:13.256243 (+147112us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:13.259899 (+  3656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.261748 (+  1849us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:13.424073 (+162325us) raft_consensus.cc:1685] finished
0709 12:14:13.424102 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.424386 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39150,"raft.queue_time_us":40805,"raft.run_cpu_time_us":48,"raft.run_wall_time_us":48,"spinlock_wait_cycles":3643264,"thread_start_us":123266,"threads_started":1,"wal-append.queue_time_us":197299}
I20260709 12:14:13.373513 25994 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:13.299361 26009 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:13.442468 26211 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:13.512317 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.512271 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:13.513613 23315 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.486244 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:13.489053 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.458837 23462 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:13.490792 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.490726 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.457967 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:13.369731 25920 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:13.444097 26027 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:13.493199 26217 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:13.493670 26016 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.494377 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:13.497614 26206 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.341665 26179 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.448418 26059 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:13.501969 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.502913 26166 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:13.503183 26119 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.485949 25994 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.559638 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.504518 26117 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.509042 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.510227 26139 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:13.545351 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.510177 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.549906 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.514166 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:13.567096 24011 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:13.550767 23313 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:13.551388 23319 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.568235 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.549315 24009 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:13.555426 23463 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:13.566427 23321 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:13.514727 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1377) took 1111 ms. Trace:
I20260709 12:14:13.557852 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.559173 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:13.504125 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.572347 23470 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:13.572417 24010 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:13.573055 23313 raft_consensus.cc:2368] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4.
I20260709 12:14:13.561578 26059 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.575888 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.559779 23461 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:13.577793 23315 raft_consensus.cc:2410] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:13.580013 24015 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:13.560210 26166 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:13.565500 25994 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:13.584136 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:13.582986 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.581969 23317 rpcz_store.cc:276] 0709 12:14:12.403230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.403371 (+   141us) service_pool.cc:224] Handling call
0709 12:14:13.511469 (+1108098us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400148f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:13.514675 (+  3206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766002944}
I20260709 12:14:13.590456 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.587689 23319 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
W20260709 12:14:13.592983 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 101452us
I20260709 12:14:13.593484 24011 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:13.596909 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:13.597618 23317 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.597956 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:13.555007 26016 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.582649 24012 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:13.595165 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:13.600816 23313 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.598470 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.557016 26206 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.602610 23317 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:13.599128 26219 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:13.616109 26219 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.616531 26219 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:13.619424 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.604598 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.621181 23313 raft_consensus.cc:2376] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3. Re-sending same reply.
I20260709 12:14:13.609561 26194 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:13.624588 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.611263 26016 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:13.614234 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.625301 23320 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7.
I20260709 12:14:13.625929 23463 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:13.626184 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.626747 23320 raft_consensus.cc:2376] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5. Re-sending same reply.
I20260709 12:14:13.622443 23317 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:13.629786 26164 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:13.603782 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.630864 23315 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7.
I20260709 12:14:13.631060 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 773) took 1081 ms. Trace:
I20260709 12:14:13.631721 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.632395 23315 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
W20260709 12:14:13.615684 26223 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:14:13.634313 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.634946 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:13.634786 26128 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:13.635944 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.636171 26128 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:13.605559 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:13.622409 26079 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:13.633234 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.627506 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.556203 26027 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.620107 25994 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:13.648079 23320 raft_consensus.cc:2376] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5. Re-sending same reply.
I20260709 12:14:13.648226 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:13.738643 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.648595 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.743808 23317 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.636507 23321 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:13.555645 26157 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.743165 23315 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.650369 24010 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:13.754673 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 149216us
I20260709 12:14:13.755695 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:13.765076 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.789314 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:13.794778 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.770813 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:13.802606 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:13.584329 26139 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.852419 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.853420 23321 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:13.745105 26224 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:14:13.851284 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:13.644354 26227 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:13.851188 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:13.855113 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:13.855348 23470 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:13.618575 26217 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:13.665764 26133 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:13.857537 23317 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:13.784624 26103 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:13.806150 26184 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:13.860085 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 63298us
I20260709 12:14:13.805891 26164 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:13.744709 25920 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:13.944372 26164 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:13.944938 26164 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:13.640069 26169 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:13.929793 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.934196 23396 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:14:13.943589 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1862) took 1221 ms. Trace:
I20260709 12:14:13.645030 26098 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:13.850878 26137 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:13.958978 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.960326 23317 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:13.997421 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.930796 26027 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:13.931231 26117 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.806568 26119 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 12:14:14.006855 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:54652 (request call id 45) took 1499 ms. Trace:
I20260709 12:14:13.628258 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:13.970994 23320 rpcz_store.cc:276] 0709 12:14:12.721975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.851818 (+1129843us) service_pool.cc:224] Handling call
0709 12:14:13.942653 (+ 90835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.942866 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.942903 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:13.942956 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:13.943003 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.943385 (+   382us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.943519 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:13.940586 26103 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.005291 23317 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:14:14.029919 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.965s (SENT) [suppressed 41 similar messages]
I20260709 12:14:14.014235 23314 rpcz_store.cc:276] 0709 12:14:12.479429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.485265 (+  5836us) service_pool.cc:224] Handling call
0709 12:14:13.978132 (+1492867us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a966e853 00007ff0ae63c60a 00007ff0a7bc27d4 00007ff0a7bc2762 00007ff0a7bc2691 00007ff0a7bc260c 00007ff0a7bc18c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:13.978258 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408701696}
W20260709 12:14:14.056695 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 254194us
I20260709 12:14:14.072813 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.073830 23319 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:14.084052 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.085248 24013 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:14.101595 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.102648 23393 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:14.109886 26179 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.110864 26179 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:14.111716 26179 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:14.112303 26179 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:14.114009 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.114264 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:13.856225 26166 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.115883 26166 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:14.116277 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:14.130956 26139 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.000957 26137 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:14.132833 26137 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.144035 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.146256 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.146507 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.148567 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.149390 24011 raft_consensus.cc:2368] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
W20260709 12:14:14.150635 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.165966 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.167029 24015 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:13.986001 26184 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.165079 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:14.172530 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.173022 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:13.631494 24007 rpcz_store.cc:276] 0709 12:14:12.549188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.551603 (+  2415us) service_pool.cc:224] Handling call
0709 12:14:13.630889 (+1079286us) spinlock_profiling.cc:243] Waited 80.2 ms on lock 0x7b54004e3254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:13.631001 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184536832}
I20260709 12:14:14.184656 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.053416 26194 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:14:14.053951 26117 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.152187 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:14.177789 26073 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.006s	sys 0.039s
I20260709 12:14:14.197816 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.198928 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.201220 23319 raft_consensus.cc:2410] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:14:14.205755 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.067970 26103 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.198323 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.222458 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.225821 24012 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:14.227326 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.233461 24015 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:14.239208 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:14.243080 23391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.023s	sys 0.004s
W20260709 12:14:14.244031 23391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.023s	sys 0.004s
W20260709 12:14:14.140808 26016 consensus_queue.cc:1175] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 2 index: 5 } }
I20260709 12:14:14.241995 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:14.256698 26167 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.078s	user 0.008s	sys 0.002s
I20260709 12:14:14.248836 23389 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:14.273201 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.260593 25994 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.147598 26179 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:14.277947 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.286355 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 256469us
W20260709 12:14:14.294005 23434 leader_election.cc:336] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.062s (SENT)
I20260709 12:14:14.294489 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.294452 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:14.287420 26219 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
I20260709 12:14:14.299268 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:14.299644 26098 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.300832 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:14.300978 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.285025 26133 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:14.277263 26167 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.301545 26234 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.304836 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:14.306075 23464 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:14.307505 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.304142 26133 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.308133 23466 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:14.310555 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:14.305080 26192 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.850s	user 0.010s	sys 0.000s
W20260709 12:14:14.316860 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.301960 26119 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.302140 26099 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:14.322531 23464 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.323889 23394 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:14.324389 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.305850 26169 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.351475 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.352177 23464 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:14.327059 26098 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:14.366698 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.302099 26219 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.367595 23466 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.380169 26117 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.358291 26119 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.382952 26119 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.383539 23470 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.374051 26098 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.388242 26098 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.390060 26167 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.346509 26192 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.392010 26184 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.393137 26192 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.395429 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.396276 23391 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:14.363771 26137 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.397781 26137 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.398921 23389 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:14.399895 26219 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.400446 26137 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.356022 26099 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.402380 26099 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:14.403374 26128 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.002s
I20260709 12:14:14.409883 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.410579 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.411217 23464 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.411803 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.310171 26234 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.412982 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.413651 23464 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:14.373986 26236 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.416687 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.371706 26110 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:14.417309 23466 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.376348 26212 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:14.421675 23393 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:14.418278 23470 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:14.420970 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.415768 26099 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.432246 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:14.435053 26059 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:14.440863 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.414734 26164 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.421588 26212 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:14.447654 23318 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:14.448573 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.449688 23319 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:14.451155 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.452044 23315 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:14.454262 26161 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:14.442736 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.454720 26161 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:14.457823 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1846) took 2486 ms. Trace:
I20260709 12:14:14.456681 23391 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:14.459890 23316 rpcz_store.cc:276] 0709 12:14:11.971436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.971624 (+   188us) service_pool.cc:224] Handling call
0709 12:14:11.994907 (+ 23283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.218860 (+223953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.218904 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:12.218960 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:12.219022 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.457631 (+2238609us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:14.457665 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.457776 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514445696}
I20260709 12:14:14.457228 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.458740 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.461789 23314 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:14.462317 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.424317 26110 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.464627 23434 leader_election.cc:400] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4.
I20260709 12:14:14.465068 23434 leader_election.cc:403] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:14:14.466446 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.445506 26166 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.468348 23434 leader_election.cc:400] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7.
I20260709 12:14:14.421764 26236 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.472124 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.463990 26128 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:14.442188 26099 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.488463 26236 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.458048 26184 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.489574 26184 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.490566 26099 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.492201 23462 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:14.473906 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.474503 26117 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:14.474908 23287 connection.cc:256] RPC call timeout handler was delayed by -3.26390359800007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 12:14:14.474874 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:14.499080 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2477035980000437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 12:14:14.499831 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2351035980000233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
I20260709 12:14:14.477713 26128 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:14.479502 26234 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.466372 26119 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4.
I20260709 12:14:14.501736 26059 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:14.507241 26059 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.502837 26117 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.509197 26184 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.509598 26117 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.491533 26166 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.502776 26128 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:14.512768 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.480674 26212 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.475001 26161 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.497014 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.500335 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:14.477082 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.521175 23316 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.476379 26133 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:14.507807 26059 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:14.483222 26110 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.513454 23320 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:14.515352 23462 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.510087 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1475) took 1784 ms. Trace:
I20260709 12:14:14.532943 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.517747 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:14.500713 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1964035979999608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:14:14.538350 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1676035979999142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
I20260709 12:14:14.521770 23391 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:14.538954 23287 connection.cc:256] RPC call timeout handler was delayed by -3.1469035980001081s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 12:14:14.539453 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0803035980000004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260709 12:14:14.523520 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:14.537673 23317 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:14.539757 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9156035979999615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1902s for a response
W20260709 12:14:14.548137 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9102035979999528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 12:14:14.548738 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8319035980000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 12:14:14.549302 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8157035980000273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 12:14:14.550092 26119 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:14.524052 26234 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:14.554093 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.554095 25950 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:14.554960 26184 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.556638 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.557950 26165 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:14.558363 26165 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:14.559245 26148 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.559202 26165 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.559863 26148 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.560359 26165 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:14.561017 23391 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:14.539829 26161 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:14.562397 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.563927 26148 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.564893 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.565497 26212 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:14.565776 23320 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:14:14.566437 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7986035979999997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
I20260709 12:14:14.505863 26099 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:14.568022 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7887035979999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 12:14:14.571089 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7086035980000815s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 12:14:14.572244 23287 connection.cc:256] RPC call timeout handler was delayed by -2.6168035979999331s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 12:14:14.572631 23287 connection.cc:256] RPC call timeout handler was delayed by -2.5142035979999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2348s for a response
W20260709 12:14:14.573107 23287 connection.cc:256] RPC call timeout handler was delayed by -2.4269035980000808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2445s for a response
W20260709 12:14:14.573819 23287 connection.cc:256] RPC call timeout handler was delayed by -2.0822035979999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2828s for a response
W20260709 12:14:14.574492 23287 connection.cc:256] RPC call timeout handler was delayed by -1.564703598000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.34030000000000005s for a response
I20260709 12:14:14.573972 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:14.568851 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.574635 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 578) took 2240 ms. Trace:
I20260709 12:14:14.523262 26166 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.576159 23320 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:14.575039 26119 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.575384 23312 rpcz_store.cc:276] 0709 12:14:12.334726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.346032 (+ 11306us) service_pool.cc:224] Handling call
0709 12:14:12.346333 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.740365 (+394032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.574399 (+1834034us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54002751d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:14.574572 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859835776}
I20260709 12:14:14.578042 26184 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:14.579339 23387 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:14.579435 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.537173 23388 rpcz_store.cc:276] 0709 12:14:12.726129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.726281 (+   152us) service_pool.cc:224] Handling call
0709 12:14:12.726812 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.727093 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:13.228264 (+501171us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.509522 (+1281258us) spinlock_profiling.cc:243] Waited 896 us on lock 0x7b1800321700. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:14:14.509735 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.509789 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:14.509843 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.509911 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.510029 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20079,"prepare.queue_time_us":608586,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2073,"raft.queue_time_us":1589,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":2061184,"thread_start_us":593717,"threads_started":2,"wal-append.queue_time_us":263509}
I20260709 12:14:14.580291 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:14.582267 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.538412 26211 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.984s	user 0.007s	sys 0.007s
I20260709 12:14:14.595137 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.569962 26164 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:14.607632 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.608009 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.686594 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.609709 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.611387 26157 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:14.589912 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.618692 26080 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:14.694304 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.690047 23464 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:14.612077 23320 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:14.638918 23315 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:14.632092 26209 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:14.627238 23391 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:14.645569 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1792) took 2090 ms. Trace:
I20260709 12:14:14.648515 26139 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.654803 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.731355 23466 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
W20260709 12:14:14.656401 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.621999 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1492) took 1107 ms. Trace:
I20260709 12:14:14.733732 23396 rpcz_store.cc:276] 0709 12:14:13.514311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.514481 (+   170us) service_pool.cc:224] Handling call
0709 12:14:13.556981 (+ 42500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.970080 (+413099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.621760 (+651680us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54003458d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:14.621937 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866899200}
I20260709 12:14:14.623977 23313 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:14.736120 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.736825 24011 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:14.530917 26133 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:14.682169 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102837us
I20260709 12:14:14.682871 23318 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:14.682821 23316 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:14.683161 23312 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:14.683425 23314 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:14.570890 26099 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.620010 26206 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.716521 23319 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:14.634711 26059 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.689234 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:14.723325 23317 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:14.723488 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.620028 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 687) took 1165 ms. Trace:
I20260709 12:14:14.775076 26099 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:14.724462 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:14.783283 23320 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:14.725114 26110 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:14.784855 26139 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.777980 26156 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:14:14.785746 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.690310 26211 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.787048 26139 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:14.788249 26211 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:14.726095 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:14.789357 26234 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:14.790136 23462 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
W20260709 12:14:14.728804 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 148607us
I20260709 12:14:14.790915 24016 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:14.727852 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.791132 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:14.738375 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:14.740826 23315 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:14.791690 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63947.7us
I20260709 12:14:14.779742 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.793195 24014 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.739821 26175 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:14.688997 26236 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:14.726802 26027 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.700s	user 0.003s	sys 0.008s
W20260709 12:14:14.790052 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 64028.6us
I20260709 12:14:14.799607 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.725258 26209 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:14.809939 23395 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260709 12:14:14.810844 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 72580.3us
I20260709 12:14:14.780026 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.814683 24015 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:14.819617 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.798388 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.820276 24010 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:14.820627 24011 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:14.728731 26117 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.800829 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:14.771778 26133 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:14.839028 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.803841 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:14.808305 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.820091 23470 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:14.823972 24009 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:14.824705 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:14.827344 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:14.844646 23462 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:14.845155 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:14.825920 26146 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:14.818658 26219 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:14:14.795562 26080 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.846962 25660 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:14.827248 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.827462 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 696) took 1370 ms. Trace:
I20260709 12:14:14.838903 26234 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:14:14.846835 26010 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:14.804234 26024 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:14.867429 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.862732 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:14.796065 24007 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:14.846437 23362 leader_election.cc:400] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9.
I20260709 12:14:14.877020 23362 leader_election.cc:403] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:14.724416 25948 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.856112 23396 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:14.856264 24016 rpcz_store.cc:276] 0709 12:14:13.457001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.559302 (+102301us) service_pool.cc:224] Handling call
0709 12:14:14.827269 (+1267967us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400398fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:14.827405 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2819409152}
I20260709 12:14:14.857703 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.854828 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:14.881464 24015 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:14.881747 23395 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:14.865135 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:14.837390 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.799616 26009 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.877790 24014 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:14.799909 26249 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:14.811223 25950 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:14.851783 26146 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.826571 26175 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:14.913465 23980 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:14.892149 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.889267 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:14.851608 26027 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.776050 26103 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.851747 24012 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:14.913632 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 587) took 1891 ms. Trace:
I20260709 12:14:14.827252 26194 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:14:14.924239 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77920.4us
I20260709 12:14:14.928884 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:14.727283 23461 rpcz_store.cc:276] 0709 12:14:12.554882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.580231 (+ 25349us) service_pool.cc:224] Handling call
0709 12:14:12.631757 (+ 51526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:13.581993 (+950236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.645236 (+1063243us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b5400186fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:14.645479 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926988544}
I20260709 12:14:14.858865 26010 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:14.858248 26234 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:14.917393 26024 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:14.882926 26139 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:14.864316 26212 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:14:14.938578 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:14.847910 26117 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.894542 26072 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.920974 26146 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:14.929768 23321 rpcz_store.cc:276] 0709 12:14:13.021862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.931074 (+909212us) service_pool.cc:224] Handling call
0709 12:14:13.984014 (+ 52940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.984270 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:14.285329 (+301059us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.349204 (+ 63875us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.635243 (+286039us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b1800214ae0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:14:14.913115 (+277872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:14.913191 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:14.913273 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.913368 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.913550 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":426950,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":5091,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":6970,"spinlock_wait_cycles":14140672,"thread_start_us":600882,"threads_started":3,"wal-append.queue_time_us":104527}
I20260709 12:14:14.879832 26165 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:14:14.947144 24014 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:14.948977 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.949075 23392 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:14.923254 25884 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9.
I20260709 12:14:14.930035 26249 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.963106 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:14.888900 25948 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:14.962075 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.958432 24011 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:14.911404 26009 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:14.944084 26156 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:14.942185 26234 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:14.972353 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:14.985222 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.758s (SENT)
I20260709 12:14:14.985970 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:14.944201 26139 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:14.999075 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:14.791513 24008 rpcz_store.cc:276] 0709 12:14:13.454239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.455215 (+   976us) service_pool.cc:224] Handling call
0709 12:14:13.493348 (+ 38133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.629591 (+136243us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:13.629632 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:13.629691 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:13.629745 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.619758 (+990013us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54006bdad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:14.619809 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.619959 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4459,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":288880000}
W20260709 12:14:15.020673 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 179084us
I20260709 12:14:15.021665 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:15.028779 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56562.2us
I20260709 12:14:15.029631 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:15.000582 26128 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.038033 23387 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:15.038931 26192 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:15.039175 26128 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:15.037205 26080 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:15.040470 26080 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:14.997967 25681 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:15.044885 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.000288 26059 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.048099 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:14.931377 25950 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.048749 23321 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:15.010604 26139 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:15.042775 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 491) took 1280 ms. Trace:
I20260709 12:14:15.009675 26249 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:14.962348 26072 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.047924 25660 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.022086 26236 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:15.055548 23463 rpcz_store.cc:276] 0709 12:14:13.762288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.762469 (+   181us) service_pool.cc:224] Handling call
0709 12:14:13.813145 (+ 50676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:14.270831 (+457686us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:14.270875 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.487543 (+216668us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:14.487974 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.726026 (+238052us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:14.726058 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.726145 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.042431 (+316286us) raft_consensus.cc:1685] finished
0709 12:14:15.042458 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.042703 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162819,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":24215,"spinlock_wait_cycles":43170176,"thread_start_us":673980,"threads_started":2,"wal-append.queue_time_us":455788}
I20260709 12:14:15.063618 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.021857 26127 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:14:15.089531 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.020545 26024 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.004952 26175 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.059006 25681 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.057276 26137 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:14:15.074867 26256 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:15.041420 26080 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.045168 26128 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:15.095175 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:15.088896 26252 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.085s	user 0.001s	sys 0.001s
I20260709 12:14:15.068045 25660 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.089416 26249 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:15.118605 23321 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:15.048273 26010 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.135176 26128 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.154668 26103 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.155119 25660 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.087505 26234 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.162367 26234 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:15.163558 26255 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:14:15.166133 26009 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:15.169891 26165 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.169374 26139 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.170392 26165 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.171478 25948 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.172518 26165 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:15.174307 25950 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.176328 26117 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.177075 26117 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:15.148413 26128 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.184031 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.193552 24015 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:15.184759 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:15.211005 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.242157 23394 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:15.242745 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:15.242780 23394 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.243726 23315 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:15.177997 26139 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.255062 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:15.255869 23320 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.258388 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.250972 23394 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.262580 23394 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:15.264884 26171 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:15.273093 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 273934us
I20260709 12:14:15.274808 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:15.243127 26133 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.304944 23394 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:15.313154 23394 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:15.317193 23389 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:15.281344 26192 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:15.317636 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 254130us
I20260709 12:14:15.306267 26173 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:15.320640 26173 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.321096 26133 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:15.272893 26171 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.322659 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1491) took 1963 ms. Trace:
I20260709 12:14:15.321578 26173 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.329206 26171 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.266654 25681 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.338052 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.347049 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.281091 26099 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.347952 23387 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:15.350747 23387 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.352542 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.348785 26119 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:14:15.354590 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.355216 26119 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:15.355643 26119 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:15.356662 26119 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.357282 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.347908 26110 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:14:15.358273 26249 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:15.309527 26179 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:14:15.355712 23396 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:15.366591 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.370154 26256 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:15.368211 26260 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:15.378074 26117 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:14:15.378404 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.378793 26117 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:15.379505 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:15.380122 26117 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:15.382460 26119 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:15.382584 23394 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:15.382860 26157 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.654s	user 0.004s	sys 0.007s
I20260709 12:14:15.383584 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.383628 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.384354 23394 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:15.384830 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.385154 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.385957 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.386224 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.386790 23394 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:15.387061 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.387539 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.388079 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.389110 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.390054 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.390305 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:15.391002 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.391841 26128 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.395661 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.396314 26256 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:15.401815 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:15.403223 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:15.403790 23470 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:15.404193 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:15.373834 23387 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.414202 23387 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:15.406692 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.407567 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.415481 23393 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:15.410210 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.416025 23389 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:15.416249 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:15.414446 24013 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:15.417258 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:15.418344 24016 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:15.420984 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.421775 23465 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:14:15.422132 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.423013 23463 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:15.423688 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.424887 23470 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:15.406680 23396 raft_consensus.cc:2376] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6. Re-sending same reply.
I20260709 12:14:15.429366 23391 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:15.430385 23470 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:15.431059 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:15.431806 24011 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:15.438313 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.439389 23393 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:14:15.443262 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:15.444003 24014 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.447482 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.449299 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.450073 23470 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:15.451359 23393 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:15.452450 26267 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:15.453105 26266 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:15.434862 26260 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:15.457259 23392 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:15.421130 26225 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.460763 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:15.485831 24015 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:15.432701 26176 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:15.415373 26256 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:15.464344 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.474994 26099 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:15.434645 26249 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:15.476207 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:15.477504 26146 consensus_meta.cc:287] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.002s
I20260709 12:14:15.477967 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:15.463770 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:15.478899 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:15.455727 26128 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.474889 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.488533 26176 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.496052 24014 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.498148 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.506480 23389 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:15.507586 23465 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.440587 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:15.494007 26256 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:15.493947 24012 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:15.428697 26192 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.495487 26225 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.482460 26266 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.482800 26110 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:15.521672 23466 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:15.518996 23389 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:14:15.534060 24015 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:15.558975 24009 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:15.561126 23396 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:15.513797 23387 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:15.561678 23396 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.545125 23465 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.548599 23470 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:15.551981 24011 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:15.562752 23396 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.574416 23396 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.580013 23462 raft_consensus.cc:2410] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:15.519042 23461 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.508615 26146 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.600454 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.602481 26146 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:15.540668 26266 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.605851 24010 raft_consensus.cc:2368] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
W20260709 12:14:15.618938 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.620051 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:15.623354 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.623899 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.627085 23465 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:15.627055 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:15.634532 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:15.635334 23461 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 2.
I20260709 12:14:15.635778 23396 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:14:15.636296 23396 pending_rounds.cc:85] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 5
I20260709 12:14:15.636562 23396 pending_rounds.cc:107] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 6.6
W20260709 12:14:15.637313 26268 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:14:15.636873 23396 raft_consensus.cc:2889] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:15.643590 26256 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:15.647521 26139 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:15.649307 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:15.649910 26146 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.650301 23312 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:15.647801 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:15.650465 26146 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.652613 26146 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:15.652809 26236 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:15.653365 26236 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:15.611073 26209 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.654522 26209 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:15.654474 26236 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.656980 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:15.657627 23315 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:14:15.658457 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:15.659016 23463 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:14:15.521929 26176 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.628541 26234 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:15.674628 23463 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:15.678455 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:15.681250 23312 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:15.669148 26072 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:15.680624 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:15.637521 26172 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:15.694885 23980 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:15.699383 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 282361us
I20260709 12:14:15.700083 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:15.700373 26266 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.704627 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:15.726588 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 302951us
I20260709 12:14:15.730551 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:15.727738 24013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 536) took 1584 ms (client timeout 1901 ms). Trace:
W20260709 12:14:15.733543 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.734997 24013 rpcz_store.cc:269] 0709 12:14:14.143522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.143694 (+   172us) service_pool.cc:224] Handling call
0709 12:14:15.727531 (+1583837us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003cb754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:15.727666 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2862607232}
W20260709 12:14:15.737885 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.696053 26156 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:15.693714 26172 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:15.739570 26110 raft_consensus.cc:2764] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:15.751593 23393 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:15.751413 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.705631 26161 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:14:15.761015 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61002.6us
I20260709 12:14:15.772950 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.776122 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.777388 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.778786 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:15.787937 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.788425 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.766023 26156 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.771023 26192 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:15.788878 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 304044us
I20260709 12:14:15.782712 26267 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:15.782495 26260 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:15.780304 26249 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:15.780386 26164 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:15.798097 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 601) took 1106 ms. Trace:
W20260709 12:14:15.794507 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 169780us
I20260709 12:14:15.820865 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:15.842751 23312 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.842396 23318 rpcz_store.cc:276] 0709 12:14:14.691872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.762364 (+ 70492us) service_pool.cc:224] Handling call
0709 12:14:14.794927 (+ 32563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.266208 (+471281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:15.266241 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:15.266284 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:15.266320 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.797828 (+531508us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:15.797870 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.798012 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083367680}
I20260709 12:14:15.821688 23314 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:14:15.822196 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.845548 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:15.818651 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.790012 26099 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:15.818998 26267 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:15.833974 26280 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:14:15.834805 26272 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:14:15.835301 26172 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:15.789682 26161 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:15.845413 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66707.9us
W20260709 12:14:15.915772 23362 connection.cc:256] RPC call timeout handler was delayed by -1.4335545359999742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1563s for a response
W20260709 12:14:15.852963 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.817356 26110 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:15.325697 23390 rpcz_store.cc:276] 0709 12:14:13.358933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.359120 (+   187us) service_pool.cc:224] Handling call
0709 12:14:13.364467 (+  5347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:13.378722 (+ 14255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:13.378769 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:13.454566 (+ 75797us) log.cc:844] Serialized 52 byte log entry
0709 12:14:14.658317 (+1203751us) spinlock_profiling.cc:243] Waited 48.1 ms on lock 0x7b1800469de0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:14:14.957696 (+299379us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:15.311000 (+353304us) log.cc:844] Serialized 14 byte log entry
0709 12:14:15.316351 (+  5351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.316459 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.316522 (+    63us) raft_consensus.cc:1685] finished
0709 12:14:15.322352 (+  5830us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.322578 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13705,"prepare.queue_time_us":208183,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":87842,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":48807,"spinlock_wait_cycles":110547200,"thread_start_us":610048,"threads_started":2,"wal-append.queue_time_us":1616447}
W20260709 12:14:15.846783 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.961759 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:15.794927 26156 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:14:15.916985 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.956317 23362 connection.cc:256] RPC call timeout handler was delayed by -1.4200545359999523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 12:14:15.970634 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2265545359998669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
I20260709 12:14:15.957130 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.851107 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:15.971120 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2085545360000651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 12:14:15.908684 26271 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.305s	user 0.000s	sys 0.004s
I20260709 12:14:15.814767 26274 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:15.828290 26249 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:15.987072 26266 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:15.988947 26249 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:15.890376 26279 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:14:15.964180 23388 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:15.972493 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:15.962605 26110 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:15.795060 26192 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.973011 26290 raft_consensus.cc:2764] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:15.879314 26165 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 12:14:15.879199 26211 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:15.978376 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 127377us
I20260709 12:14:16.002238 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:16.003487 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:15.979028 26206 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:15.989993 26179 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:15.982144 26282 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:14:15.989795 26274 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:15.909992 26269 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 12:14:15.827425 26256 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:15.844798 26260 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:15.848429 26146 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.849772 26072 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.144s
I20260709 12:14:15.860680 23312 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:15.841373 26234 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.147s
I20260709 12:14:15.973054 26172 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.957132 26267 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:15.848683 26164 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.003085 26119 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.005887 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:16.008591 26290 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:16.006887 24014 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:16.007191 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:16.009830 23318 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:16.011020 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:16.017562 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 315) took 15106 ms (client timeout 15000 ms). Trace:
W20260709 12:14:16.039377 23217 rpcz_store.cc:269] 0709 12:14:00.910890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.911074 (+   184us) service_pool.cc:224] Handling call
0709 12:14:16.017422 (+15106348us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:11.531505 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:11.823247 (+291742us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:11.823311 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:14:11.823338 (+    27us) tablet.cc:662] Decoding operations
0709 12:14:12.383862 (+560524us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:12.383944 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:12.383980 (+    36us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:14:12.385288 (+  1308us) tablet.cc:701] Row locks acquired
0709 12:14:12.385336 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:14:12.385525 (+   189us) write_op.cc:270] Start()
0709 12:14:12.385683 (+   158us) write_op.cc:276] Timestamp: P: 1783599252385458 usec, L: 0
0709 12:14:12.385707 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:14:12.386444 (+   737us) log.cc:844] Serialized 6695 byte log entry
0709 12:14:13.862344 (+1475900us) op_driver.cc:464] REPLICATION: finished
0709 12:14:14.405792 (+543448us) write_op.cc:301] APPLY: starting
0709 12:14:14.405925 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:14.406394 (+   469us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:14.406582 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:14.409128 (+  2546us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:14.420139 (+ 11011us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:14:14.424160 (+  4021us) write_op.cc:312] APPLY: finished
0709 12:14:14.508100 (+ 83940us) log.cc:844] Serialized 113 byte log entry
0709 12:14:14.749479 (+241379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:14.749728 (+   249us) write_op.cc:454] Released schema lock
0709 12:14:14.749852 (+   124us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:14.750442 (+   590us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:14.750482 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":937,"reported_tablets":29,"spinlock_wait_cycles":41984,"child_traces":[["op",{"apply.queue_time_us":453210,"apply.run_cpu_time_us":66548,"apply.run_wall_time_us":344634,"num_ops":16,"prepare.queue_time_us":905369,"prepare.run_cpu_time_us":18220,"prepare.run_wall_time_us":1054123,"raft.queue_time_us":399813,"raft.run_cpu_time_us":27756,"raft.run_wall_time_us":746667,"replication_time_us":1260891,"thread_start_us":1234951,"threads_started":5,"wal-append.queue_time_us":660036,"wal-append.run_cpu_time_us":28862,"wal-append.run_wall_time_us":1758714}]]}
I20260709 12:14:15.998214 26192 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:15.995476 26110 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:16.023759 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51321.9us
I20260709 12:14:16.006359 26209 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:16.045307 26209 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.045791 26209 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:16.006454 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:16.036561 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:16.050036 26059 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:16.050200 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43831.2us
I20260709 12:14:16.051476 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:16.052213 26059 raft_consensus.cc:2764] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:16.054081 26110 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.058832 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:16.064976 26256 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:16.065572 26256 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:16.066620 26256 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.073211 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:16.017562 26274 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:16.075085 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:16.075133 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:16.077658 23388 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:16.077919 23464 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:16.078482 23470 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:16.082471 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:16.085418 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:16.032351 26278 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.087446 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:16.088210 23395 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:16.089123 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:16.094007 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.094265 23390 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:16.096500 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.096751 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.097229 23312 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:16.069502 24012 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:16.102458 26266 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:16.099643 23313 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:16.100098 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:16.103382 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.103670 23314 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:16.103910 24009 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:16.104146 23312 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:16.018301 26179 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.109587 23319 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:14:16.069482 23218 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 3, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.112165 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.032090 26265 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:14:16.079872 26290 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:16.113790 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.115499 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:16.115794 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 87465us
I20260709 12:14:16.116608 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.120985 23320 raft_consensus.cc:2368] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
I20260709 12:14:16.116963 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.122160 23321 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:16.123082 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.125041 23321 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:14:16.113371 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.095669 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:16.138434 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.104584 26165 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:16.069926 26119 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.033500 26146 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.145123 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.156613 23316 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:16.156646 23321 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:16.146595 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.036463 26161 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.151005 26119 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:16.148485 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.153096 26234 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:16.154788 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.157903 23313 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 3: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:16.159292 26110 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.161656 23461 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:16.162112 26146 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.162531 24010 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:16.103149 26236 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:14:16.168689 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.114989 26174 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:14:16.170012 23320 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:16.175657 23218 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 5 to 7, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 7 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.113855 26273 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:14:16.184513 23287 leader_election.cc:400] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4.
I20260709 12:14:16.192303 24010 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:16.167867 26171 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:16.155561 26173 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 12:14:16.205957 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.206508 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.174962 26119 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.216061 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:16.236109 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:16.238638 23218 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.242215 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:16.243360 23464 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:16.171721 26234 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.247160 23470 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:16.201903 26137 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:16.250265 24009 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:16.251296 24015 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:16.251375 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:16.252406 26291 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.260931 23464 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:16.255409 23218 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:16.178059 26209 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.246809 26266 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:16.278271 24009 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:16.254082 26137 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:16.211678 26171 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.213227 26119 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.288949 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.267599 25660 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:16.306264 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 92598.6us
I20260709 12:14:16.267608 26110 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:16.306864 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:16.317507 23316 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:16.317756 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:16.322782 24015 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:16.316236 23218 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 5, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 5 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:16.311350 23321 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:16.290503 26164 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:16.337482 23321 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:14:16.339792 23320 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:16.338174 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:16.379706 26209 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:16.326707 26256 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:16.381821 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 43722.2us
I20260709 12:14:16.382802 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:16.389542 26225 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.391873 26119 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:16.392316 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.394294 26179 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:16.394457 26234 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:16.307605 26080 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:14:16.396731 26137 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:16.402040 23434 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.965s (SENT)
I20260709 12:14:16.270466 26291 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.416656 26291 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.419597 24007 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
W20260709 12:14:16.425503 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.322149 25660 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.103473 26278 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.429021 26137 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:16.399708 26117 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:16.142458 26260 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.237694 26290 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.341595 26273 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:14:16.429412 26309 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:16.417335 23461 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:16.381230 26027 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:14:16.512182 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:16.530613 23464 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:16.530794 25660 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:16.531232 26156 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.001s
I20260709 12:14:16.531721 26156 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.399572 26293 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:16.521806 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 139084us
I20260709 12:14:16.534086 24008 raft_consensus.cc:2368] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4.
I20260709 12:14:16.534821 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:16.535902 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:16.536662 26256 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:16.537927 26293 raft_consensus.cc:2764] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:16.538550 26293 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:16.538982 26293 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:16.539848 26293 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.544433 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:16.547498 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:16.523164 26304 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.093s	user 0.000s	sys 0.010s
W20260709 12:14:16.515229 26305 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.115s	user 0.004s	sys 0.000s
I20260709 12:14:16.552444 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:16.550732 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:16.557833 23462 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:16.558180 23464 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:16.564936 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:16.566618 23464 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:16.523003 26161 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:16.585171 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.600485 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:16.603255 26172 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.569s	user 0.008s	sys 0.001s
W20260709 12:14:16.604403 26307 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:14:16.600644 26211 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 12:14:16.601207 24014 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:16.565141 26157 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.643049 23436 leader_election.cc:400] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
I20260709 12:14:16.643641 23436 leader_election.cc:403] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:16.631114 26305 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:14:16.659874 26311 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
W20260709 12:14:16.663524 26206 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.656s	user 0.007s	sys 0.005s
I20260709 12:14:16.664119 26206 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:16.665894 26171 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:16.671180 26156 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.559001 26309 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.637778 26157 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.732375 26157 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:16.732614 26110 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:16.733125 26284 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:14:16.734408 23313 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:16.736178 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:16.756801 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.638797 26211 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:16.846068 24009 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:16.598654 26161 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:16.729317 26309 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:16.630963 26172 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.211s
I20260709 12:14:16.862121 26110 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:16.892526 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.894235 24013 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:16.896611 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:16.898765 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.899192 23321 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:16.903276 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:16.903805 23321 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:16.895723 26309 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.916333 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:16.918286 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 273774us
W20260709 12:14:16.911361 26274 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.001s
I20260709 12:14:16.899168 26171 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:16.925915 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 381550us
I20260709 12:14:16.920648 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:16.898705 26110 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:16.916793 26267 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.835s	user 0.008s	sys 0.002s
I20260709 12:14:16.959635 23321 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
W20260709 12:14:16.919770 26192 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.875s	user 0.012s	sys 0.001s
I20260709 12:14:16.955072 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:16.968544 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47955.5us
I20260709 12:14:17.102176 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:16.928994 26309 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.952381 26274 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.965233 26267 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.961774 26171 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:16.976679 26110 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.056470 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 718) took 1028 ms. Trace:
I20260709 12:14:17.118270 24011 rpcz_store.cc:276] 0709 12:14:16.027803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.028460 (+   657us) service_pool.cc:224] Handling call
0709 12:14:16.028835 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.029108 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:16.483700 (+454592us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.847740 (+364040us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.848342 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.850904 (+  2562us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:16.869351 (+ 18447us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.912727 (+ 43376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.912830 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.056222 (+143392us) raft_consensus.cc:1685] finished
0709 12:14:17.056249 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.056402 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36523,"prepare.queue_time_us":143146,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":48305,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":11392,"spinlock_wait_cycles":16669056,"thread_start_us":686104,"threads_started":2,"wal-append.queue_time_us":428164}
W20260709 12:14:17.058305 26249 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.069s	user 0.011s	sys 0.002s
I20260709 12:14:17.120797 23313 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:16.924677 26311 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:14:17.053649 26314 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:17.059993 26291 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.061287 26161 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:17.066215 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.075917 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 712) took 1070 ms. Trace:
I20260709 12:14:17.078719 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:16.926184 26256 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:17.093264 23982 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.031s	sys 0.080s
W20260709 12:14:17.065304 26137 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.003s
I20260709 12:14:17.106777 23321 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:14:17.113674 25950 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:17.124495 24010 rpcz_store.cc:276] 0709 12:14:16.005160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.005650 (+   490us) service_pool.cc:224] Handling call
0709 12:14:17.075756 (+1070106us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b540011d5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:17.075855 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295129984}
I20260709 12:14:17.124529 23315 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:17.128085 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 49456us
W20260709 12:14:17.128821 23982 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.032s	sys 0.080s
W20260709 12:14:17.155761 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 597667us
W20260709 12:14:17.156586 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 2.116s (SENT)
I20260709 12:14:17.158823 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:17.163405 23982 leader_election.cc:336] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 2.201s (SENT)
I20260709 12:14:17.163882 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:17.165941 26174 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:17.166378 26267 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.166571 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.167145 26267 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:17.171948 23466 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:17.173575 25950 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:17.158350 26256 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.178848 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:17.179414 23466 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:14:17.179571 26256 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.180224 26274 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.105106 26192 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.182266 26192 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:17.175922 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:17.179697 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.183314 23470 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
W20260709 12:14:17.183847 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.175625 26171 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:17.184957 23320 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:17.182565 26309 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:17.185977 24009 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:17.194155 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.196205 23321 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:14:17.196564 23321 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:14:17.196732 23316 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:17.197098 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.130759 26176 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:17.206174 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:17.206893 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.127372 26249 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.213092 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.213938 23394 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:17.196818 23321 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:17.199137 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.200202 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.217617 24011 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:17.219257 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.207620 23390 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:17.202940 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.192205 26146 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:17.223896 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.191717 26161 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:17.195850 26139 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.013s	sys 0.032s
I20260709 12:14:17.187471 26276 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:17.187654 26301 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:17.218429 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.204296 26267 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.129550 26291 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.240347 23318 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:17.241883 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:17.217150 26174 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:17.246879 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:17.187861 26323 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.115s
I20260709 12:14:17.225596 26256 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:17.255266 23312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.000s	sys 0.026s
I20260709 12:14:17.252856 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:17.234396 26301 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:17.204206 26128 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:17.258517 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:17.223254 26219 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:14:17.229717 26192 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:14:17.234257 26276 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:17.227555 26313 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:17.257227 26146 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:17.259714 26211 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:17.258671 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1448) took 1143 ms. Trace:
I20260709 12:14:17.239945 26161 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.262198 26157 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:17.281836 23312 rpcz_store.cc:276] 0709 12:14:16.115431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.131434 (+ 16003us) service_pool.cc:224] Handling call
0709 12:14:16.131797 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.132035 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.132062 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.132658 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.517832 (+385174us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:16.686217 (+168385us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.258246 (+572029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.258352 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.258407 (+    55us) raft_consensus.cc:1685] finished
0709 12:14:17.258426 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.258587 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570321,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":46,"raft.queue_time_us":219816,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5499,"thread_start_us":1079919,"threads_started":3,"wal-append.queue_time_us":350777}
I20260709 12:14:17.290339 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:17.246636 26267 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.264950 26156 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:17.313575 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1910) took 1312 ms. Trace:
I20260709 12:14:17.314123 23314 rpcz_store.cc:276] 0709 12:14:16.001252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.002234 (+   982us) service_pool.cc:224] Handling call
0709 12:14:16.002531 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.104141 (+101610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.313242 (+1209101us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:17.313489 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232106240}
I20260709 12:14:17.314898 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:17.315508 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:17.315675 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 360633us
I20260709 12:14:17.316262 23314 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:17.316587 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:17.276422 26146 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.281800 26211 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.318105 26156 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:17.333938 26293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.006s	sys 0.013s
W20260709 12:14:17.335920 26293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.006s	sys 0.013s
W20260709 12:14:17.335237 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 126322us
I20260709 12:14:17.339612 23470 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:14:17.340049 23470 pending_rounds.cc:85] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 4
I20260709 12:14:17.340318 23470 pending_rounds.cc:107] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:14:17.340607 23470 raft_consensus.cc:2889] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:17.342952 24009 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:17.343638 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:17.345618 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:17.353421 24016 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:17.368245 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:17.368961 23462 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:17.260121 25950 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.327687 26146 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:17.378018 23469 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:17.330073 26211 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.377205 23466 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:17.367270 23314 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:17.332788 26156 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:17.390609 23465 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:17.389462 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 116692us
W20260709 12:14:17.386353 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 124247us
I20260709 12:14:17.386686 23314 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:17.388190 24014 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:17.409770 26139 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.410516 26139 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.414433 26139 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:17.365864 26293 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:17.440248 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:17.447216 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 130750us
I20260709 12:14:17.448627 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:17.452225 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:17.453673 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48139.8us
I20260709 12:14:17.446300 23314 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.467837 23470 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:17.473227 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:17.321208 26301 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.482101 23314 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.483127 26249 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.487640 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:17.420266 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:17.494738 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:17.489691 26307 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:14:17.467057 26275 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.478463 26323 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:17.530453 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:17.536077 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:17.371829 26276 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.477699 26333 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:14:17.541200 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 147235us
I20260709 12:14:17.548092 23317 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:14:17.557848 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.514559 24012 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:17.471881 26293 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.503332 26332 raft_consensus.cc:2764] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:14:17.548431 26322 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 12:14:17.526242 26291 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:17.575618 26276 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.577527 26323 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:17.543102 26249 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:14:17.637298 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 142962us
I20260709 12:14:17.649998 23314 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:14:17.650579 23314 pending_rounds.cc:85] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 3
I20260709 12:14:17.659027 23314 pending_rounds.cc:107] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:17.659391 23314 raft_consensus.cc:2889] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:17.628319 26161 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.772862 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:17.542519 26275 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.648422 26333 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:17.642921 26267 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:17.775532 26341 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:14:17.650033 26313 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.771301 26346 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:17.807832 26274 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:17.808264 26274 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:17.808545 26347 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:14:17.809067 26274 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.789501 26311 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:17.811597 24008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 715) took 1786 ms (client timeout 2028 ms). Trace:
I20260709 12:14:17.810097 26274 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:17.813395 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 277425us
I20260709 12:14:17.813608 26176 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:17.651288 26352 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.170s
W20260709 12:14:17.815418 26337 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 12:14:17.794827 26355 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:17.822757 26156 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.801903 26291 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
W20260709 12:14:17.825953 24008 rpcz_store.cc:269] 0709 12:14:16.025663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.025813 (+   150us) service_pool.cc:224] Handling call
0709 12:14:17.811209 (+1785396us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54006bce54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:17.811513 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113032576}
I20260709 12:14:17.829021 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:17.793191 26249 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.643457 26323 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.866884 26176 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:17.866977 26355 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.789783 26365 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:17.867689 26355 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.867835 26365 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:17.868337 26365 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:17.868486 26156 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.869643 26249 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.869481 26176 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.857620 26311 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.871224 26176 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.872542 26249 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:17.872699 26311 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:17.854035 26334 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.873632 26334 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:17.826086 26346 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:17.874863 26346 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.875744 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:17.876786 23316 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:17.877285 26323 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.870265 26352 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:17.887359 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:17.903842 26275 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:17.905124 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:17.893766 26352 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.914391 26352 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:17.915036 26355 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.918576 26334 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.906289 26293 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.925089 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:17.926456 24008 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:17.926837 26211 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.506s	user 0.002s	sys 0.006s
I20260709 12:14:17.928452 26355 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:17.930850 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.932724 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:17.934360 26278 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:17.937893 24011 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:17.937863 23388 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:17.896509 26311 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.940801 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:17.941169 24009 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:18.050836 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.944151 24015 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:14:17.947999 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.051909 23388 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:17.826963 26024 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.812543 26313 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:17.961814 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.961196 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.964002 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:17.969698 26331 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:17.973166 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:17.976102 26275 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:14:17.976554 26335 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.499s	user 0.001s	sys 0.001s
I20260709 12:14:17.981986 26139 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.989921 26117 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:17.996239 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.115459 24016 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:17.997838 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:17.999353 26332 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:18.000031 23434 connection.cc:256] RPC call timeout handler was delayed by -1.3070568710002135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 12:14:18.118916 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.119343 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.119704 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.120018 23434 connection.cc:256] RPC call timeout handler was delayed by -1.1920728140000847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2484s for a response
W20260709 12:14:18.120417 23434 connection.cc:256] RPC call timeout handler was delayed by -1.1682295420000628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2666s for a response
W20260709 12:14:18.120800 23434 connection.cc:256] RPC call timeout handler was delayed by -1.1555887330000587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 12:14:18.003525 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.121898 23391 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:18.005309 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1913) took 1718 ms. Trace:
I20260709 12:14:18.006369 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1904) took 2182 ms. Trace:
I20260709 12:14:18.122896 23318 rpcz_store.cc:276] 0709 12:14:16.287250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.292853 (+  5603us) service_pool.cc:224] Handling call
0709 12:14:16.316683 (+ 23830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.242413 (+925730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.005096 (+762683us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b5400149454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:18.005255 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123809152}
I20260709 12:14:18.010679 24012 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:18.014858 23317 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:18.015564 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:18.121362 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0647295420001228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2781s for a response
I20260709 12:14:18.019896 26192 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:18.020759 26219 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:14:18.021939 23312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.000s	sys 0.003s
W20260709 12:14:18.132320 23312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.001s	sys 0.004s
I20260709 12:14:18.022364 23316 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
W20260709 12:14:18.132650 23395 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.000s	sys 0.028s
W20260709 12:14:18.027976 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.058668 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:17.948343 26146 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
W20260709 12:14:18.133689 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.091651 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:18.158258 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66713.5us
I20260709 12:14:18.162065 26329 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:18.163061 26329 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:14:18.164685 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 191631us
I20260709 12:14:18.166163 26172 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Committing config change with OpId 4.5: config changed from index 3 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:18.092422 26293 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:18.008611 26098 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.019269 26127 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:18.125483 23396 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:18.169145 26293 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.169286 26098 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.171322 26293 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.172277 26276 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.172468 26098 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.117185 26139 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.174124 23321 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:18.174484 23321 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:18.175009 26098 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:18.175448 26139 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:18.175539 26128 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:18.176083 26128 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.176712 26266 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:18.177170 26128 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.178140 26276 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:18.178474 26128 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.179607 26276 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:18.182103 26146 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.183162 24008 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 12:14:18.184198 26146 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:18.126231 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0637887329999103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1617s for a response
W20260709 12:14:18.185470 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0322887330000867s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 12:14:18.123062 23315 rpcz_store.cc:276] 0709 12:14:15.824353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.824547 (+   194us) service_pool.cc:224] Handling call
0709 12:14:15.840996 (+ 16449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.158541 (+1317545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.006085 (+847544us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540001e354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:18.006300 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2951324032}
W20260709 12:14:18.022091 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 478815us
I20260709 12:14:18.186771 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.187498 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:18.187547 26297 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:18.188375 26174 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:18.188782 26174 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.188961 26297 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:18.189396 26297 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.189946 26297 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.189651 26174 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.190726 26174 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.191026 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.191731 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.192319 23464 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.192476 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:18.156116 26334 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:18.189320 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.188961 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:18.156392 23395 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.003s	sys 0.028s
I20260709 12:14:18.193939 23462 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:18.193706 26334 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.175505 26098 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.194882 26098 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.195442 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.199640 23434 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.195834 26334 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.189745 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.202165 26355 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.202663 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.202898 26139 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:18.203619 26301 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.203477 26139 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.204061 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.205022 26161 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:18.205708 26161 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.205756 26355 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:18.206131 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.207304 26355 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:18.113229 23392 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:18.208240 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:18.210704 26146 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:18.211578 26206 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:18.212375 26362 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:18.212730 26362 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:18.213577 26362 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.214710 26362 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.221912 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:18.223292 26098 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.223796 26355 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.224225 26161 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.228226 26161 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.231714 26161 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.233774 26139 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.234916 26139 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:18.235666 26139 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:18.225284 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.225672 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.238574 26334 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:18.226780 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.239639 26334 consensus_queue.cc:1243] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:18.239933 24015 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:18.240407 26333 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:18.240818 26333 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:18.227593 26355 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.242033 26333 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.243273 26355 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:18.248247 26333 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:18.250720 26098 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:18.256578 26249 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:18.259738 26249 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:18.261211 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:18.262163 26249 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:18.262622 26249 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.265705 26249 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:18.271956 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.282415 26355 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.283241 26355 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.048753 26173 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:14:18.290237 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:18.290848 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.291455 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.292074 23318 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:18.292335 26173 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:18.292718 26173 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.292874 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.291579 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1920) took 1166 ms. Trace:
I20260709 12:14:18.294703 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1525) took 1110 ms. Trace:
I20260709 12:14:18.293644 23313 rpcz_store.cc:276] 0709 12:14:17.125153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.125284 (+   131us) service_pool.cc:224] Handling call
0709 12:14:17.125649 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:17.125978 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:17.126005 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:17.126536 (+   531us) log.cc:844] Serialized 52 byte log entry
0709 12:14:17.266901 (+140365us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:17.266953 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.267889 (+   936us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.975827 (+707938us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:18.291286 (+315459us) raft_consensus.cc:1685] finished
0709 12:14:18.291307 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.291509 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":54947,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":95,"spinlock_wait_cycles":15503744,"thread_start_us":136000,"threads_started":1,"wal-append.queue_time_us":244988}
I20260709 12:14:18.295109 23395 rpcz_store.cc:276] 0709 12:14:17.184016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.184226 (+   210us) service_pool.cc:224] Handling call
0709 12:14:17.203770 (+ 19544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.203973 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:17.387249 (+183276us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.294191 (+906942us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.294359 (+   168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:18.294410 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:18.294458 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.294532 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.294647 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99748,"prepare.queue_time_us":400818,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":111,"thread_start_us":945654,"threads_started":2}
I20260709 12:14:18.296326 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.300034 26173 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.302968 26173 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.307094 26313 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.115935 26331 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:18.296326 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.308857 26331 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.309948 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.311589 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.312875 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.082831 26267 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:18.313795 26267 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.314324 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.315079 23318 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 2.
I20260709 12:14:18.315315 23313 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:18.316290 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.316426 26267 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:18.322355 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61222.6us
W20260709 12:14:18.322945 24010 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.002s	sys 0.045s
I20260709 12:14:18.328982 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:18.334236 24010 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.002s	sys 0.045s
I20260709 12:14:18.056828 26211 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.059501 26311 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.115003 26117 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:18.083329 26024 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.325155 23313 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:14:18.356086 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.372390 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.373216 23313 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:18.374514 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:18.374807 23982 leader_election.cc:336] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.630s (SENT)
I20260709 12:14:18.375488 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.368849 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.377842 23464 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:18.287559 26321 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s	user 0.004s	sys 0.019s
W20260709 12:14:18.382498 23320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.013s	sys 0.018s
W20260709 12:14:18.386276 23320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.013s	sys 0.019s
I20260709 12:14:18.391331 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 512) took 1869 ms. Trace:
I20260709 12:14:18.390177 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:18.393024 24014 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:18.395777 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64246us
W20260709 12:14:18.384429 26321 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.004s	sys 0.020s
I20260709 12:14:18.132335 26351 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.273s
I20260709 12:14:18.401008 23469 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:18.403542 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.406400 23462 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.367195 26161 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.403399 23321 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.404067 23464 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:18.412227 23321 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:18.361544 26110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.008s	sys 0.009s
I20260709 12:14:18.351624 26367 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:18.422165 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:18.425860 23464 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:14:18.422624 26110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.009s	sys 0.009s
I20260709 12:14:18.428545 26256 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.428982 26256 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:18.431102 26211 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.383020 26301 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:14:18.400481 26364 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.435967 23462 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:18.436721 26367 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:18.392194 23463 rpcz_store.cc:276] 0709 12:14:16.522301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.522452 (+   151us) service_pool.cc:224] Handling call
0709 12:14:16.527337 (+  4885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.527525 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:16.527796 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.969369 (+1441573us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b18000e00e0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0ad06c3c7 00007ff0a967e9a7 00007ff0a968e0bf 00007ff0a968e038
0709 12:14:18.367451 (+398082us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.390420 (+ 22969us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:18.390488 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:18.390548 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.391115 (+   567us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.391265 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":427886,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":27221,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":99810,"spinlock_wait_cycles":126738176,"thread_start_us":555062,"threads_started":2,"wal-append.queue_time_us":715202}
I20260709 12:14:18.343685 26276 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:14:18.133371 26332 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.350462 26352 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.452942 26367 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.463617 26297 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.453343 26276 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:18.466234 26276 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.467259 26276 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.468057 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 728) took 1234 ms. Trace:
I20260709 12:14:18.468405 26276 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:18.469959 26355 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.468648 24010 rpcz_store.cc:276] 0709 12:14:17.233665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.235500 (+  1835us) service_pool.cc:224] Handling call
0709 12:14:17.235913 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.246244 (+ 10331us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:17.444633 (+198389us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.334658 (+890025us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.335254 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.335442 (+   188us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:18.342075 (+  6633us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.352278 (+ 10203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.352388 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:18.467809 (+115421us) raft_consensus.cc:1685] finished
0709 12:14:18.467835 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:18.467986 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20860,"prepare.queue_time_us":573257,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":224817,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":36651,"thread_start_us":1057882,"threads_started":3,"wal-append.queue_time_us":372919}
I20260709 12:14:18.470971 26352 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.472514 26024 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.419907 26117 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.423416 26161 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:18.447948 26346 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.020s
W20260709 12:14:18.449717 26365 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.000s
I20260709 12:14:18.487110 26249 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.403654 26219 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:14:18.489163 26332 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.501088 26311 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:18.502028 26161 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:18.486234 26346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.015s	sys 0.020s
I20260709 12:14:18.495761 26365 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.390671 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:18.624630 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.626405 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 292668us
I20260709 12:14:18.627055 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.629017 23313 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:18.630208 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.630844 23314 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:18.630741 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.632516 23313 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:18.400928 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.636019 23318 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
W20260709 12:14:18.440105 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.638376 23319 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:18.640589 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 462647us
W20260709 12:14:18.647186 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.654739 26110 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:14:18.497259 26256 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.564936 26363 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:18.679240 26363 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.679920 26172 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52264 (request call id 91) took 12181 ms. Trace:
I20260709 12:14:18.681981 26256 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.684990 26360 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:18.685806 26360 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:18.686255 24007 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.016s	sys 0.023s
W20260709 12:14:18.686781 24007 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.118s	user 0.016s	sys 0.023s
I20260709 12:14:18.546332 26297 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.687253 23388 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:14:18.505626 25950 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.006s	sys 0.030s
I20260709 12:14:18.687453 26360 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:18.688548 25950 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.007s	sys 0.031s
I20260709 12:14:18.689488 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.689849 26365 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.690636 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:18.691047 26360 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.692538 23321 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:14:18.693145 23321 pending_rounds.cc:85] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:14:18.693511 23321 pending_rounds.cc:107] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:14:18.697052 23321 raft_consensus.cc:2889] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:18.657060 26139 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:18.721601 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1926) took 1496 ms. Trace:
I20260709 12:14:18.722136 23321 rpcz_store.cc:276] 0709 12:14:17.225235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.231285 (+  6050us) service_pool.cc:224] Handling call
0709 12:14:17.232047 (+   762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.174029 (+941982us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b54001633a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:14:18.402999 (+228970us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b4c00313100. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7940a20 00007ff0a7951a29 00007ff0a7917644 00007ff0a793ef2d 00007ff0a793eb93 00007ff0a966f18b 00007ff0a966fcb5 00007ff0a966af30 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416
0709 12:14:18.721092 (+318093us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.721482 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351465600}
W20260709 12:14:18.721801 26327 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 12:14:18.721051 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.017s
I20260709 12:14:18.685063 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:18.730664 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:18.734426 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:18.735370 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:18.736619 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51991.2us
I20260709 12:14:18.737027 26297 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:18.737560 26297 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.737975 26297 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.738287 26290 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:18.739179 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.739243 26290 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:18.741535 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.007s	sys 0.006s
W20260709 12:14:18.741914 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.008s	sys 0.006s
I20260709 12:14:18.742427 26176 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:18.742892 26176 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.743285 26176 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:18.743830 26315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.520s	user 0.003s	sys 0.025s
W20260709 12:14:18.744274 26315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.568s	user 0.003s	sys 0.025s
W20260709 12:14:18.747061 26260 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.167s	user 0.010s	sys 0.015s
W20260709 12:14:18.747634 26260 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.219s	user 0.010s	sys 0.015s
W20260709 12:14:18.741932 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.748734 26182 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.001s	sys 0.011s
W20260709 12:14:18.749300 26182 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.001s	sys 0.011s
I20260709 12:14:18.751570 26117 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:18.751470 26297 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:18.742439 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 726922us
I20260709 12:14:18.764171 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:18.765240 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.735883 23319 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:18.767132 26387 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:18.767500 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.010s	sys 0.017s
I20260709 12:14:18.693899 26363 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.772435 26387 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.773983 26363 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:18.774096 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 397216us
I20260709 12:14:18.765162 26333 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:18.777382 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.777873 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:18.778239 23469 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:18.778692 23463 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:18.778328 26333 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.780041 26297 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:18.782117 26260 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.782852 26260 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:18.782166 26393 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:14:18.786530 26249 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.788447 26260 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.790778 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.791626 24010 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:18.794199 23982 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.720s (SENT)
I20260709 12:14:18.794281 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.797597 24010 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:18.796531 23469 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:18.798826 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.799993 26380 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:18.800561 26380 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:18.801013 26380 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:18.805528 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.808413 23388 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:18.810262 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.812636 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.815696 23393 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:18.818463 23320 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:14:18.820770 26333 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.827386 26333 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:18.827687 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.824837 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.828306 26277 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:18.829265 26277 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.830324 26367 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:18.830269 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.832002 26277 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.680646 26172 rpcz_store.cc:276] 0709 12:14:06.498183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.498345 (+   162us) service_pool.cc:224] Handling call
0709 12:14:08.411247 (+1912902us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4c001e3a80. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7940a20 00007ff0a7951a29 00007ff0a7917644 00007ff0a793ef2d 00007ff0a793eb93 00007ff0a95cd1af 00007ff0a95ccf3e 00007ff0a9652888 00007ff0a966f75c 00007ff0a967076a 00007ff0a966f91b
0709 12:14:08.419127 (+  7880us) log.cc:844] Serialized 508 byte log entry
0709 12:14:18.679589 (+10260462us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":242,"mutex_wait_us":31211,"raft.queue_time_us":56428,"raft.run_cpu_time_us":1171,"raft.run_wall_time_us":1171,"spinlock_wait_cycles":43734272,"thread_start_us":61959,"threads_started":1,"wal-append.queue_time_us":75981,"wal-append.run_cpu_time_us":1854,"wal-append.run_wall_time_us":1001881}
I20260709 12:14:18.833271 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:18.835398 26110 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.835577 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:18.836745 23391 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 4: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:18.836304 26110 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.840113 23319 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:18.841456 26110 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.842494 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:18.843649 23316 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:18.844444 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 644833us
I20260709 12:14:18.845970 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:18.846849 23321 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.846513 26211 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:18.848354 26211 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:18.848793 26211 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:18.850340 26127 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:18.851009 26127 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.842500 26367 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:18.852017 26127 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.765523 26192 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.854934 26192 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.853084 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:18.854588 26127 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:18.854992 26211 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.857847 26367 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:18.858286 26192 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.858433 26367 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:18.860935 26211 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:18.883819 26367 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.884764 26172 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.885464 26172 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.829232 26333 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.869684 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 22fc2096e78f47158bbb4bae3a474e2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:14:18.872571 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:18.872996 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:18.876653 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.877831 23394 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:18.881533 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:18.896785 24010 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:18.896773 23467 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:18.909409 23316 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:18.911795 23321 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:14:18.926220 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.933404 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51987.9us
W20260709 12:14:18.936864 23390 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.009s	sys 0.003s
I20260709 12:14:18.933492 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.950902 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.951920 26315 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:18.953960 26146 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:18.954280 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:18.954613 23387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.013s	sys 0.008s
I20260709 12:14:18.954982 23394 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:18.957614 26182 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:18.958144 26315 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:18.958779 26182 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:18.959187 26182 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:18.959290 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:18.959836 24010 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:14:18.960394 26315 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:18.960446 26182 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:18.909409 26277 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 12:14:18.962347 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.955029 23387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.013s	sys 0.008s
I20260709 12:14:18.936136 26387 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.967818 23467 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:18.969481 26290 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:18.970847 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:18.970896 26176 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:18.973767 24009 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:18.977015 26333 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:18.981982 23469 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:14:18.983057 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.989272 26146 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:18.990589 26146 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.015914 26290 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:19.016678 26290 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.019057 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 801) took 1652 ms. Trace:
I20260709 12:14:19.019315 26290 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.015985 26355 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 12:14:19.016120 26174 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.821s	user 0.016s	sys 0.000s
I20260709 12:14:19.022549 26380 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.023883 23389 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:19.024937 26380 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.026057 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 473) took 1054 ms. Trace:
I20260709 12:14:19.026616 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.027328 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 472) took 1056 ms. Trace:
I20260709 12:14:19.027401 23388 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:19.028499 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.027758 23387 rpcz_store.cc:276] 0709 12:14:17.970828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.060446 (+ 89618us) service_pool.cc:224] Handling call
0709 12:14:18.060748 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.117208 (+ 56460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:18.117252 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.131711 (+ 14459us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.979344 (+847633us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:18.979814 (+   470us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.026950 (+ 47136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.027047 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.027084 (+    37us) raft_consensus.cc:1685] finished
0709 12:14:19.027096 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.027245 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133064,"prepare.queue_time_us":180,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":93892,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3671,"thread_start_us":622608,"threads_started":1,"wal-append.queue_time_us":700058}
I20260709 12:14:19.029549 26297 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:19.029137 23391 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:19.033973 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.034406 26182 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:19.025215 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1533) took 1012 ms. Trace:
I20260709 12:14:19.036921 23389 rpcz_store.cc:276] 0709 12:14:18.012774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.110548 (+ 97774us) service_pool.cc:224] Handling call
0709 12:14:18.285912 (+175364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.024266 (+738354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.024933 (+   667us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54001e3f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:19.025145 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695928576}
I20260709 12:14:19.039474 26290 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:19.039994 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.040221 26290 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.040959 24008 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:19.044117 24013 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:14:19.045063 24010 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:19.002008 26176 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:19.047886 26405 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.128s	user 0.002s	sys 0.000s
W20260709 12:14:19.048684 26401 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:14:19.057039 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.057374 23467 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:19.056962 23470 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:19.050979 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
W20260709 12:14:19.059670 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43589.5us
I20260709 12:14:19.067114 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:19.061523 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 102440us
I20260709 12:14:19.074970 26290 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:19.075742 24009 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:19.011875 26315 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:19.076538 26315 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.080013 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.076963 23314 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:19.084010 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:19.099079 26346 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:19.099920 26346 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:19.105728 26139 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.106411 26139 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.107676 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.108760 26139 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.114454 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.115326 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.115473 23467 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:19.115595 25950 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:19.117566 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1930) took 1829 ms. Trace:
I20260709 12:14:19.117856 26379 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:19.101212 26346 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.026449 23390 rpcz_store.cc:276] 0709 12:14:17.971349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.060457 (+ 89108us) service_pool.cc:224] Handling call
0709 12:14:18.117299 (+ 56842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.117674 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:18.289981 (+172307us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.977488 (+687507us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.978280 (+   792us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.978454 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:18.978895 (+   441us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.025628 (+ 46733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.025756 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.025804 (+    48us) raft_consensus.cc:1685] finished
0709 12:14:19.025815 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.025983 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47596,"prepare.queue_time_us":176031,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":107,"raft.queue_time_us":91303,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":4041,"thread_start_us":647027,"threads_started":1,"wal-append.queue_time_us":423248}
I20260709 12:14:19.117100 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.124749 26182 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:19.121505 23982 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 2.556s (SENT) [suppressed 15 similar messages]
I20260709 12:14:19.132793 23467 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:19.136060 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.118331 26379 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:19.140412 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47417.7us
I20260709 12:14:19.143878 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:19.144919 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:19.145576 23464 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:19.146323 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.019508 24007 rpcz_store.cc:276] 0709 12:14:17.366573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.368030 (+  1457us) service_pool.cc:224] Handling call
0709 12:14:17.441013 (+ 72983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.489656 (+ 48643us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:18.687524 (+1197868us) log.cc:844] Serialized 14 byte log entry
0709 12:14:18.994956 (+307432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.009710 (+ 14754us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:19.018701 (+  8991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.018824 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.018980 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42946,"prepare.queue_time_us":323128,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":37098,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":1357950,"threads_started":2,"wal-append.queue_time_us":290235}
I20260709 12:14:19.155593 23465 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.156782 26172 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.157620 26363 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.158169 26363 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.160401 26363 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:19.161715 26182 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.118049 23312 rpcz_store.cc:276] 0709 12:14:17.251929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.323112 (+ 71183us) service_pool.cc:224] Handling call
0709 12:14:17.323421 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:18.132793 (+809372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:18.132827 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:18.286710 (+153883us) log.cc:844] Serialized 26 byte log entry
0709 12:14:18.995710 (+709000us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:19.013790 (+ 18080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.047859 (+ 34069us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:19.064442 (+ 16583us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.064492 (+    50us) raft_consensus.cc:1685] finished
0709 12:14:19.064504 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.080761 (+ 16257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16796,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":15656704,"thread_start_us":1095306,"threads_started":2,"wal-append.queue_time_us":289760}
I20260709 12:14:19.145496 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:19.181800 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 64359.2us
I20260709 12:14:19.184142 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.184317 23464 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:14:19.189154 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63380.6us
I20260709 12:14:19.163349 26379 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:19.196775 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.200155 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.208029 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.209234 26139 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:19.196935 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:19.213613 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.214988 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:19.215875 26387 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:19.216564 26387 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:19.217274 26174 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:19.217912 26174 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:19.218835 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.218796 26174 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.219959 23390 raft_consensus.cc:2410] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 8 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:19.221107 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:19.222364 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.226354 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.228030 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:19.233110 23312 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:19.191257 26315 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:19.241508 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.248162 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.249722 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.249887 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.250278 23396 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10. Re-sending same reply.
I20260709 12:14:19.250531 23387 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:19.251274 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.251477 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.191809 26172 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:14:19.251941 23396 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:19.252640 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.253283 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:19.253786 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.254179 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.254594 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.254957 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.258661 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.252820 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.255867 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.265909 23391 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:19.262938 23395 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:19.253403 23387 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:19.267330 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.267510 23388 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:19.268808 23390 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:19.257886 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:19.269965 23396 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.280320 23394 raft_consensus.cc:2410] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:14:19.279724 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:19.258037 26315 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:19.294188 26380 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:19.290360 26182 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:19.369721 23465 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:19.305104 26367 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:19.385375 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 76285.2us
W20260709 12:14:19.389595 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 134784us
W20260709 12:14:19.390523 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 132690us
I20260709 12:14:19.320641 26139 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:19.401651 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:19.408362 23391 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:19.401681 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:19.411598 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:19.417287 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.420461 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.423126 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:19.409897 26315 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.425945 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:19.427942 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:19.424595 26139 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:19.429623 26387 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:19.443943 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.441597 23318 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:19.437719 26127 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:19.447945 26387 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.436599 26315 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.486691 23313 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:19.489209 26364 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:19.495384 26276 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:19.498165 26408 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:14:19.505964 26370 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:14:19.475656 26127 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:19.513312 26127 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.531966 23396 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:19.477931 26387 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:19.547976 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 165055us
I20260709 12:14:19.554373 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.555802 23391 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:19.516599 26385 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:19.581583 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54661.6us
I20260709 12:14:19.480628 26139 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:14:19.597036 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.565006 26385 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.597998 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:19.608685 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:19.589252 26364 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.614657 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:19.617413 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.606092 26211 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:19.606520 26385 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:19.618796 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.626803 25950 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:19.627018 26274 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.627525 26274 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.629530 26380 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:19.630010 26380 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.630412 26380 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.633682 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:19.634796 24015 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:19.637704 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:19.638027 26182 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.638780 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.638859 26182 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.639716 24007 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:19.641036 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.642907 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.643970 24012 raft_consensus.cc:2410] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:14:19.644742 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.646245 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.647220 24016 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:19.654101 26375 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:14:19.619639 26276 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:19.655074 26276 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.655490 26276 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:19.657325 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:19.658354 25950 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:19.658759 25950 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:19.659122 25950 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:19.662329 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.663226 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:19.668991 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.674151 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65575.6us
I20260709 12:14:19.675312 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.676930 26315 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:19.669793 26182 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.680522 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.638120 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.674170 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:19.676981 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:19.685256 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 88309.9us
I20260709 12:14:19.699379 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.701578 26315 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:19.702068 26315 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:19.702482 26315 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:19.706394 26274 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.707623 26380 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.727888 26311 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:19.730362 26355 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:19.733639 26297 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:19.734098 26379 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.734531 26297 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.736323 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:19.737205 24011 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
W20260709 12:14:19.738442 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.742182 26413 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:14:19.743284 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.747925 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.748800 23464 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:19.752604 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 75792.9us
W20260709 12:14:19.756448 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 64964.9us
I20260709 12:14:19.756477 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:19.774260 23319 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:19.779443 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.782985 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 84576.9us
I20260709 12:14:19.620271 26364 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:19.794890 26277 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.807667 26139 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:19.798408 26313 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.810284 26313 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.812880 26313 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:19.806969 26425 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.102s	user 0.000s	sys 0.002s
I20260709 12:14:19.808629 26277 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.807323 26297 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:19.616804 26383 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.174s
I20260709 12:14:19.781854 26380 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:19.800467 26419 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:14:19.854557 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:19.799041 26211 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:19.861447 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:19.872576 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:19.862885 23462 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:19.886570 23319 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:14:19.887009 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72319.3us
I20260709 12:14:19.887312 23319 pending_rounds.cc:85] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:14:19.897045 23319 pending_rounds.cc:107] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:14:19.857775 26274 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.897351 23319 raft_consensus.cc:2889] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:19.914544 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 220717us
I20260709 12:14:19.926630 26433 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:19.926764 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 54327.1us
W20260709 12:14:19.931634 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68656.2us
I20260709 12:14:19.932585 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:19.933545 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.869052 26379 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:19.878690 26297 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:19.889513 26360 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:19.892915 26211 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:19.894744 26139 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:19.939685 26435 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:19.960073 26435 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:19.960518 26435 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:19.894932 26172 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:19.972687 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 190151us
W20260709 12:14:19.960124 26432 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.256s	user 0.001s	sys 0.001s
I20260709 12:14:19.938107 26363 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:19.978289 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 252216us
I20260709 12:14:19.944360 26182 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:14:19.988030 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50463.9us
I20260709 12:14:19.961359 26327 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:14:19.998047 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:20.006014 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:19.980244 26139 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:19.966234 26274 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:19.955597 26433 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.005415 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:20.015470 26364 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:19.971637 26360 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.031754 23470 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:20.031862 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:19.966427 26297 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.027683 26380 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:19.981853 26172 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.037529 26429 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:19.987753 26387 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.053200 23465 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
W20260709 12:14:20.065924 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73479.9us
W20260709 12:14:20.068884 26417 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:14:20.058678 23216 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:20.056218 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58265.3us
I20260709 12:14:20.026738 26146 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:14:20.057302 26364 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.082525 23396 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:20.084419 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.073839 26277 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.093158 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:20.082206 26327 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:20.094798 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.052093 26211 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.101729 24011 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:20.079742 26429 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.111088 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1946) took 1107 ms. Trace:
W20260709 12:14:20.121635 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.100589 26355 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.111624 23315 rpcz_store.cc:276] 0709 12:14:19.004112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.004284 (+   172us) service_pool.cc:224] Handling call
0709 12:14:19.064413 (+ 60129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:19.091960 (+ 27547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.092009 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.134559 (+ 42550us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.409981 (+275422us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:19.442630 (+ 32649us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.442864 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.442960 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.110841 (+667881us) raft_consensus.cc:1685] finished
0709 12:14:20.110872 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.111008 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1133,"prepare.queue_time_us":67164,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":11836,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":21360,"thread_start_us":255993,"threads_started":1,"wal-append.queue_time_us":543396}
I20260709 12:14:20.104456 26311 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:20.146315 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 165571us
I20260709 12:14:20.128629 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1849) took 1019 ms. Trace:
W20260709 12:14:20.147605 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.174194 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.110212 26327 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:20.177757 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.128012 26446 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:20.180789 26446 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:20.180617 26355 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.181239 26327 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.097745 26276 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:20.193634 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 209375us
I20260709 12:14:20.099957 26139 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.197134 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.173197 23461 rpcz_store.cc:276] 0709 12:14:19.056720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.074559 (+ 17839us) service_pool.cc:224] Handling call
0709 12:14:19.280424 (+205865us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:19.284027 (+  3603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:19.341946 (+ 57919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.341993 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.413403 (+ 71410us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.599438 (+186035us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:19.599554 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.599745 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.778109 (+178364us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:20.076019 (+297910us) raft_consensus.cc:1685] finished
0709 12:14:20.076040 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.076186 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10150,"raft.queue_time_us":19924,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":63387,"spinlock_wait_cycles":43584128,"thread_start_us":162679,"threads_started":1,"wal-append.queue_time_us":277686}
I20260709 12:14:20.199318 23391 raft_consensus.cc:2368] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9.
I20260709 12:14:20.199712 23314 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:20.202330 23314 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:20.203289 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:20.206496 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 83161.8us
I20260709 12:14:20.215205 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.183148 26355 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.180632 26274 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.219604 26450 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.220368 26449 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:20.220844 26363 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:20.222000 26438 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:14:20.181246 26446 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.217978 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.231986 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.192251 26440 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:20.237497 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 253824us
I20260709 12:14:20.217516 26277 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:20.213251 26447 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:20.232380 26172 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:20.276975 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.257900 26436 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 12:14:20.236239 26363 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:20.242767 26313 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:20.281006 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 63099.9us
I20260709 12:14:20.293356 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.236796 26433 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.252952 26387 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.258595 26447 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.286155 23314 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.232136 26297 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.259881 26311 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:20.277696 24009 raft_consensus.cc:2955] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:20.234455 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:20.236542 26450 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:20.311972 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53444.5us
I20260709 12:14:20.315305 23393 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.317258 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:20.318027 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62956.9us
I20260709 12:14:20.319921 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:20.321544 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:20.322132 24016 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.291918 26435 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.292811 26363 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:20.310851 26433 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:14:20.330090 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:20.330744 24011 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:20.331584 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.332870 23465 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:20.304319 26172 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:20.342151 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.344625 23314 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.309629 26447 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:20.345815 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 92707.4us
I20260709 12:14:20.346029 23469 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:20.329989 26387 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.353830 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:20.335685 26172 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:20.343871 26385 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.004s
I20260709 12:14:20.363410 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.364064 23468 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:20.366235 26444 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:14:20.362147 26172 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.368167 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 802) took 1295 ms. Trace:
I20260709 12:14:20.368721 24014 rpcz_store.cc:276] 0709 12:14:19.037358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.042238 (+  4880us) service_pool.cc:224] Handling call
0709 12:14:19.042705 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:19.068092 (+ 25387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.068147 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.103838 (+ 35691us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.515227 (+411389us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800348160. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:14:19.828359 (+313132us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:19.828416 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.828537 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.317968 (+489431us) raft_consensus.cc:1685] finished
0709 12:14:20.317987 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.332307 (+ 14320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19460,"raft.queue_time_us":35208,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":14470,"spinlock_wait_cycles":39740160,"thread_start_us":312863,"threads_started":1,"wal-append.queue_time_us":924423}
I20260709 12:14:20.375195 23468 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:20.349525 26429 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.353394 26440 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:14:20.384811 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53263.7us
I20260709 12:14:20.392853 23314 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 12:14:20.393362 23314 pending_rounds.cc:85] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:14:20.393749 23314 pending_rounds.cc:107] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:14:20.377141 26311 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.394021 23314 raft_consensus.cc:2889] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:20.376379 26276 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.393152 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:20.396941 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61371.6us
I20260709 12:14:20.398304 26311 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:20.402889 26315 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.001s
I20260709 12:14:20.403375 26315 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.406891 26434 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:20.407375 26434 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.407698 26434 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.408378 26127 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:20.384646 26440 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:20.409912 26440 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:20.358445 26387 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:20.424777 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80306.8us
I20260709 12:14:20.361420 26456 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:14:20.426302 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:20.426460 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:20.432456 26435 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:20.432957 26457 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:14:20.434706 26387 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.439410 24016 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:20.440084 26450 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.440781 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:20.467067 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.467962 26327 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.468376 26327 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.468736 26327 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.469106 26274 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:20.469568 26274 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:20.469941 26274 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:20.373212 26385 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.475250 26385 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.476385 26367 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:20.452728 26360 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.478273 26327 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.479416 23466 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:20.481088 26327 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:20.487377 26442 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.264s	user 0.000s	sys 0.003s
I20260709 12:14:20.442477 26429 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:20.443127 26355 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:20.489082 26355 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:20.490141 26434 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.490335 26355 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.492159 26434 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.431655 26311 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.450109 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.451351 23465 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:14:20.452382 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.495661 23467 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:20.480855 26447 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:20.459774 25950 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.001s
I20260709 12:14:20.460162 26456 consensus_queue.cc:1243] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:14:20.464210 23434 connection.cc:256] RPC call timeout handler was delayed by -1.36973784099996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30510000000000004s for a response
W20260709 12:14:20.465605 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.525353 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 175726us
I20260709 12:14:20.531042 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.532737 23314 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:20.534353 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.505187 26315 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:20.501322 26445 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.541457 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.481271 26363 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.551755 26146 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:20.556550 23317 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:20.553212 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:20.537262 26297 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.519623 26450 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:14:20.567093 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:20.569485 23314 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.543244 26364 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:20.520089 25950 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.570408 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:20.563894 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 95220us
I20260709 12:14:20.534436 26172 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:20.582468 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 175918us
I20260709 12:14:20.568270 26276 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.588094 23468 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:20.565254 26182 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.551972 26445 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.587347 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:20.583793 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:20.588964 23318 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:20.570669 26447 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.574786 26433 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.542043 26440 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.601344 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:20.650982 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 67257.2us
I20260709 12:14:20.672919 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:20.654000 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.674005 23320 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:20.683565 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:20.568087 26146 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.685140 26274 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.685730 25950 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.686053 26146 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.686669 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:20.667415 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:20.687312 23313 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.687424 23396 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.688702 23394 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:20.570221 26297 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.689955 23388 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:20.691144 26297 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:20.666276 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.665512 26360 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:20.686797 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:20.691929 26360 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.692126 23317 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:20.692373 26360 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:20.599819 26182 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.698673 23392 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.699764 26182 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.701859 26172 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.703343 26446 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.704002 25950 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:20.704689 25950 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.583068 26364 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.581863 26469 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:20.708150 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:20.739403 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66609.8us
I20260709 12:14:20.742296 23392 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
W20260709 12:14:20.653333 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 117024us
I20260709 12:14:20.713641 26462 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.744638 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 222468us
I20260709 12:14:20.745537 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:20.746704 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:20.749956 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.751989 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.753969 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:20.755894 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:20.707598 26420 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:14:20.791571 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50901.1us
W20260709 12:14:20.792290 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49040.3us
I20260709 12:14:20.724555 23396 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:20.752915 23465 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:20.770232 23314 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:14:20.713485 26463 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:14:20.687563 26274 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:20.596273 26276 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.809304 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.810118 23470 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:20.810407 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 55078.8us
I20260709 12:14:20.812754 23464 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:20.705384 26446 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:20.707500 26364 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:20.707115 25950 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:20.818696 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:20.827757 23387 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:20.816039 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:20.828922 24013 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:20.821691 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:20.829998 23465 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:20.824928 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:20.745354 26447 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.831964 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.816478 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.833806 23466 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.836117 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.837872 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:20.838478 24016 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:20.838836 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:20.840880 23463 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:20.791501 26172 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:20.837378 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:20.839828 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 241100us
I20260709 12:14:20.844583 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.843394 24010 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:20.847095 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:20.846153 23312 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:20.745070 26462 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.739276 26433 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:20.851519 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 104887us
I20260709 12:14:20.853648 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:20.860956 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:20.825670 26346 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:14:20.871764 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:20.860152 26383 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:20.841300 26472 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:14:20.861466 26174 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:20.797675 26469 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:20.856573 26447 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:20.904598 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:20.863420 25950 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:14:20.889662 26383 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.928683 23313 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:20.902802 26387 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.938901 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 86629.7us
I20260709 12:14:20.900256 26475 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:20.944344 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:20.872915 26466 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:14:20.877496 26355 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:14:20.938828 26383 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:20.926772 26470 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:20.967933 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:20.976011 24013 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:20.930696 26172 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:20.982470 23387 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:20.953176 26128 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:20.950894 26445 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:20.987160 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 120934us
I20260709 12:14:20.989416 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:20.995323 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:20.997403 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:20.945989 26387 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:21.001958 23320 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:21.003153 23320 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:20.988585 26470 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:20.987735 26128 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:21.001191 26468 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 12:14:21.014467 26481 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:20.996013 26440 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:21.671849 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 542) took 1223 ms. Trace:
W20260709 12:14:21.051788 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50142.5us
W20260709 12:14:21.428346 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.405331 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:21.616559 23466 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.004s
I20260709 12:14:21.618546 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1552) took 1017 ms. Trace:
I20260709 12:14:21.694399 23466 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:21.331277 23318 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.000s
I20260709 12:14:21.695427 23318 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:21.145098 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:21.029734 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.602947 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:21.038275 23316 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:21.445055 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.697827 23316 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:21.697984 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 95145us
I20260709 12:14:21.264822 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:21.640796 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.526974 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.699442 24011 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:21.041810 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.699592 23467 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:21.700521 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2296) took 1166 ms. Trace:
I20260709 12:14:21.042213 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.656275 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:21.702836 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:21.335440 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.175305 24013 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:14:21.634258 24012 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.000s
I20260709 12:14:21.706926 24012 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:21.596180 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 813) took 1876 ms. Trace:
I20260709 12:14:21.104259 24010 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:21.227837 23394 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:21.233155 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.700907 23467 rpcz_store.cc:276] 0709 12:14:20.534081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.526665 (+992584us) service_pool.cc:224] Handling call
0709 12:14:21.700466 (+173801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:21.692216 23468 rpcz_store.cc:276] 0709 12:14:20.448039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.449124 (+  1085us) service_pool.cc:224] Handling call
0709 12:14:20.449510 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:20.588489 (+138979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.671588 (+1083099us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54002256d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:21.671759 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318299008}
W20260709 12:14:21.712662 23980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.001s	sys 0.019s
I20260709 12:14:21.232843 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:21.770128 23359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.081s	sys 0.178s
I20260709 12:14:21.034973 23320 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:21.532902 23216 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 7, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:21.745512 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 1973) took 1117 ms. Trace:
W20260709 12:14:21.795984 23359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.097s	sys 0.186s
W20260709 12:14:21.714231 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 58101.5us
W20260709 12:14:21.673913 26480 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.685s	user 0.000s	sys 0.003s
I20260709 12:14:21.798611 23465 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:21.798830 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
W20260709 12:14:21.799762 23287 connection.cc:256] RPC call timeout handler was delayed by -1.6065360739999051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 12:14:21.802762 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5930360739998832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
W20260709 12:14:21.803295 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5255360740000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
W20260709 12:14:21.596715 26383 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.000s
I20260709 12:14:21.801003 23392 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.801352 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:21.405807 26448 raft_consensus.cc:993] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:21.796895 23318 rpcz_store.cc:276] 0709 12:14:20.579172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.579318 (+   146us) service_pool.cc:224] Handling call
0709 12:14:21.695967 (+1116649us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:21.803306 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.812633 23467 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:21.232771 26429 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:21.310110 26439 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.124s	user 0.005s	sys 0.009s
I20260709 12:14:21.799719 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:21.394659 26487 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:21.713505 26128 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:21.803788 23287 connection.cc:256] RPC call timeout handler was delayed by -1.3806360739999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 12:14:21.804490 23980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.001s	sys 0.034s
I20260709 12:14:21.305696 26447 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:21.042794 26433 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:21.860532 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:21.804266 23436 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.008s	sys 0.026s
W20260709 12:14:21.883793 23436 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.008s	sys 0.026s
W20260709 12:14:21.884167 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 587727us
I20260709 12:14:21.806663 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:21.886381 23394 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:21.887353 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.887933 23394 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:21.592353 26311 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:21.888798 26311 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:21.889710 26311 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.890686 26311 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:21.233026 26315 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.708045 24009 rpcz_store.cc:276] 0709 12:14:19.719962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.720366 (+   404us) service_pool.cc:224] Handling call
0709 12:14:19.721069 (+   703us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:20.090101 (+369032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.090144 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:20.127557 (+ 37413us) log.cc:844] Serialized 534 byte log entry
0709 12:14:20.277571 (+150014us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:20.429944 (+152373us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.522905 (+ 92961us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.581944 (+ 59039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.595571 (+1013627us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b540008e8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:21.595647 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.595850 (+   203us) raft_consensus.cc:1685] finished
0709 12:14:21.595872 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.596085 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23820,"prepare.queue_time_us":100657,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":138409,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":11519,"spinlock_wait_cycles":79397760,"thread_start_us":488189,"threads_started":3,"wal-append.queue_time_us":216805,"wal-append.run_cpu_time_us":3688,"wal-append.run_wall_time_us":1220638}
I20260709 12:14:21.893708 26315 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:21.894371 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.694537 23389 rpcz_store.cc:276] 0709 12:14:20.601217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.601612 (+   395us) service_pool.cc:224] Handling call
0709 12:14:20.665676 (+ 64064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.738875 (+ 73199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.738906 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:20.743166 (+  4260us) log.cc:844] Serialized 26 byte log entry
0709 12:14:21.600342 (+857176us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b1800390580. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1
0709 12:14:21.600529 (+   187us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:21.600585 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.600684 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.600735 (+    51us) raft_consensus.cc:1685] finished
0709 12:14:21.600744 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.618480 (+ 17736us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75805,"raft.queue_time_us":73333,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":39062,"spinlock_wait_cycles":13417984,"thread_start_us":72538,"threads_started":1,"wal-append.queue_time_us":205045}
I20260709 12:14:21.427248 26475 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.895368 26475 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.896073 26315 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:21.534449 26363 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:21.481359 26472 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.009361 26484 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:21.907784 26484 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:21.907855 23218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 341) took 25373 ms (client timeout 15000 ms). Trace:
I20260709 12:14:21.813045 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.908901 23468 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:21.021929 26471 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.956s
I20260709 12:14:21.817083 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.817826 23317 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:21.917364 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.818037 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.818542 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.818795 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.820545 23392 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:21.821653 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.918920 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:21.821938 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.023808 26360 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:21.822681 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:21.433132 26479 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.301666 26435 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.033674 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.923187 24007 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:21.828382 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.923666 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1871) took 1618 ms. Trace:
I20260709 12:14:21.306442 26469 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:21.670320 26276 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:21.744478 26211 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.863932 24016 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:21.744674 26434 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:21.018911 26387 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:21.828809 23312 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:21.898082 23463 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
W20260709 12:14:21.802763 24014 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.000s	sys 0.019s
W20260709 12:14:21.899932 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0377360739998949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
W20260709 12:14:21.952649 24014 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.000s	sys 0.019s
I20260709 12:14:21.898114 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.953387 23467 raft_consensus.cc:2368] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9.
I20260709 12:14:21.954523 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.822562 23316 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.902133 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.796191 26367 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:14:21.803659 23216 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 5 to 7, term changed from 5 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to <none>, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193), VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194), VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195), VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 6 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:21.796435 26443 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:14:21.823194 26458 raft_consensus.cc:1081] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:21.816540 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.974361 23387 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:21.012110 26277 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:21.909948 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.982896 23468 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:21.910753 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:21.914383 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.914475 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:21.801178 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:21.919098 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.920555 23393 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:21.305879 26462 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:21.926359 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.930157 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.903082 26447 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:21.931412 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:22.014369 23466 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:21.924106 24011 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:21.929764 26173 raft_consensus.cc:2764] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:21.030385 26172 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.935796 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:21.936645 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:22.026682 24012 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:21.902923 26450 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:21.938402 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:21.803515 26485 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.126s
I20260709 12:14:21.939023 23461 rpcz_store.cc:276] 0709 12:14:20.305118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.306542 (+  1424us) service_pool.cc:224] Handling call
0709 12:14:20.319331 (+ 12789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:20.319531 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:20.352072 (+ 32541us) log.cc:844] Serialized 14 byte log entry
0709 12:14:21.626796 (+1274724us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b18002661a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0ad06c3c7 00007ff0a967e9a7 00007ff0a968e0bf 00007ff0a968e038
0709 12:14:21.627002 (+   206us) log.cc:844] Serialized 14 byte log entry
0709 12:14:21.923334 (+296332us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:21.923387 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:21.923438 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.923510 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.923611 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1257028,"prepare.queue_time_us":1331798,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":85,"raft.queue_time_us":5592,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":10003712,"thread_start_us":295993,"threads_started":1}
I20260709 12:14:21.813182 26383 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:21.830448 26429 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.951206 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.958390 23320 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.960021 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:21.861346 26487 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.961534 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:21.962069 26139 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.968485 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 671) took 1442 ms. Trace:
I20260709 12:14:21.975126 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:21.984169 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:22.048739 23468 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.983146 23216 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 4 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:22.009176 24013 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:21.932133 26479 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:22.016835 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 215720us
I20260709 12:14:22.022416 26355 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:22.021040 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:22.056993 23396 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:22.060899 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:22.061736 23394 raft_consensus.cc:606] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:21.951251 26475 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:21.970100 26274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.022s	user 0.008s	sys 0.021s
I20260709 12:14:21.960429 26434 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:22.064682 23391 raft_consensus.cc:2368] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
I20260709 12:14:22.037458 24008 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:21.972661 26499 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:22.037209 23470 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:21.939328 26435 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.068571 23467 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:21.930122 26472 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.073679 23216 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:21.951457 26211 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.008810 23465 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 3: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:22.078845 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.050661 23387 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:22.048729 23463 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
W20260709 12:14:22.064675 26274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.008s	sys 0.021s
I20260709 12:14:22.047313 23319 rpcz_store.cc:276] 0709 12:14:20.525881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.526024 (+   143us) service_pool.cc:224] Handling call
0709 12:14:20.543249 (+ 17225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.543552 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.543591 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:20.549679 (+  6088us) log.cc:844] Serialized 26 byte log entry
0709 12:14:20.875082 (+325403us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:20.875121 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.875192 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.673231 (+798039us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:21.952023 (+278792us) raft_consensus.cc:1685] finished
0709 12:14:21.952041 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.968400 (+ 16359us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156441,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":7600000,"thread_start_us":318754,"threads_started":1,"wal-append.queue_time_us":382345}
I20260709 12:14:22.032508 23318 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:21.908340 23218 rpcz_store.cc:269] 0709 12:13:56.295588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.295771 (+   183us) service_pool.cc:224] Handling call
0709 12:13:57.007374 (+711603us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:13:57.368239 (+360865us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a6550640 00007ff0af427846 00007ff0af53190e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:21.669300 (+24301061us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:16.969967 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:17.084422 (+114455us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:17.084488 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:14:17.084533 (+    45us) tablet.cc:662] Decoding operations
0709 12:14:17.089650 (+  5117us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:17.094213 (+  4563us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:17.094397 (+   184us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:14:17.243139 (+148742us) tablet.cc:701] Row locks acquired
0709 12:14:17.246902 (+  3763us) write_op.cc:260] PREPARE: finished
0709 12:14:17.256853 (+  9951us) write_op.cc:270] Start()
0709 12:14:17.263916 (+  7063us) write_op.cc:276] Timestamp: P: 1783599257254582 usec, L: 0
0709 12:14:17.265827 (+  1911us) op_driver.cc:348] REPLICATION: starting
0709 12:14:17.368348 (+102521us) log.cc:844] Serialized 2089 byte log entry
0709 12:14:18.433176 (+1064828us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a966fe28 00007ff0ad0aa581 00007ff0ad0a9d58 00007ff0ad0ae9f1 00007ff0ad0ae9a9 00007ff0ad0ae939 00007ff0ad0ae901 00007ff0ad0adb2d 0000000000388634 0000000000388479
0709 12:14:18.588919 (+155743us) op_driver.cc:464] REPLICATION: finished
0709 12:14:18.999866 (+410947us) write_op.cc:301] APPLY: starting
0709 12:14:19.000088 (+   222us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:19.000457 (+   369us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:19.000509 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:19.212146 (+211637us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:19.255574 (+ 43428us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:14:19.255644 (+    70us) write_op.cc:312] APPLY: finished
0709 12:14:19.256093 (+   449us) log.cc:844] Serialized 47 byte log entry
0709 12:14:19.256348 (+   255us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:19.256497 (+   149us) write_op.cc:454] Released schema lock
0709 12:14:19.256595 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:19.257189 (+   594us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:19.257228 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1156310,"reported_tablets":27,"spinlock_wait_cycles":440008192,"child_traces":[["op",{"apply.queue_time_us":410475,"apply.run_cpu_time_us":5105,"apply.run_wall_time_us":257335,"num_ops":5,"prepare.queue_time_us":472608,"prepare.run_cpu_time_us":33168,"prepare.run_wall_time_us":1472923,"raft.queue_time_us":85435,"raft.run_cpu_time_us":40898,"raft.run_wall_time_us":367351,"replication_time_us":1322905,"spinlock_wait_cycles":7142784,"thread_start_us":1521551,"threads_started":4,"wal-append.queue_time_us":1118757,"wal-append.run_cpu_time_us":13234,"wal-append.run_wall_time_us":2029902}]]}
W20260709 12:14:22.067189 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 114230us
I20260709 12:14:22.087702 23465 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:22.043054 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:22.087910 23463 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:22.016588 26277 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.056125 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:22.090180 23388 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.090965 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:22.033586 26315 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.025262 26276 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:22.028206 26447 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.088582 23216 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 3, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:22.058930 26355 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:22.092481 23465 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:22.067778 26474 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:22.038890 26450 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:21.959652 26470 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.078171 26502 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.096611 24009 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:22.129755 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.128777 26315 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.130622 26355 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.137274 23468 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:22.150300 24011 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:22.097323 24010 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.100457 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:22.153717 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53343.9us
I20260709 12:14:22.154999 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.144572 24013 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:22.056785 26139 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:22.159593 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 125838us
I20260709 12:14:22.165036 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:22.162110 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.097713 26458 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:22.149709 26276 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:22.074707 26499 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.160612 26433 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:22.182384 24010 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:22.182349 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.149250 26469 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.167038 26274 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.183341 23216 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 5 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 6 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:22.169764 26489 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:22.207931 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.166728 26501 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:22.186831 23470 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:22.174817 26456 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:22.178651 26355 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:22.158243 26490 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.225224 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 198756us
I20260709 12:14:22.234910 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:22.181699 26458 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.185305 26387 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.229231 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50154.6us
I20260709 12:14:22.230881 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:22.243386 24011 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:22.229979 26274 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.191269 26434 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.249871 23388 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:22.239925 23216 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:22.241165 26472 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.200162 26503 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:22.251564 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.177867 26498 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:14:22.253240 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1873) took 1947 ms. Trace:
I20260709 12:14:22.253494 24013 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:14:22.223181 26433 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:22.154800 26502 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.258055 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:22.223291 26469 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.256614 23462 rpcz_store.cc:276] 0709 12:14:20.305812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.312571 (+  6759us) service_pool.cc:224] Handling call
0709 12:14:20.330141 (+ 17570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.339818 (+  9677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.339864 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:20.354569 (+ 14705us) log.cc:844] Serialized 26 byte log entry
0709 12:14:20.390592 (+ 36023us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:21.959393 (+1568801us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.252951 (+293558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.253020 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.253060 (+    40us) raft_consensus.cc:1685] finished
0709 12:14:22.253070 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.253185 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1271379,"prepare.queue_time_us":1236990,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":5214,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3099,"thread_start_us":596356,"threads_started":4,"wal-append.queue_time_us":9086,"wal-append.run_cpu_time_us":2495,"wal-append.run_wall_time_us":1154141}
I20260709 12:14:22.256006 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:22.282929 24013 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:22.268772 24007 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:22.258046 24011 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
W20260709 12:14:22.368960 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 206959us
I20260709 12:14:22.370096 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.277729 23216 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 1 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 3 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:22.249336 26448 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:22.376458 26456 raft_consensus.cc:2764] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:14:22.250138 26490 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:22.383832 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 127898us
I20260709 12:14:22.384779 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:22.432691 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 575) took 1370 ms. Trace:
I20260709 12:14:22.445127 24007 rpcz_store.cc:276] 0709 12:14:20.995796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.998459 (+  2663us) service_pool.cc:224] Handling call
0709 12:14:22.365796 (+1367337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:22.233398 26489 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:22.245527 26458 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.413926 23216 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:22.631542 26274 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.256853 26503 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:22.649823 26514 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:14:22.662269 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 277545us
W20260709 12:14:22.662472 23393 consensus_meta.cc:287] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.000s
W20260709 12:14:22.563549 23320 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.000s
I20260709 12:14:22.674722 23393 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:22.472502 24008 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:22.344419 26495 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
W20260709 12:14:22.653151 26508 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:14:22.254047 26504 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:22.690918 26503 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:22.552042 23389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.006s	sys 0.006s
W20260709 12:14:22.693003 24014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.011s	sys 0.017s
I20260709 12:14:22.680786 23320 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 12:14:22.693553 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 435548us
W20260709 12:14:22.671574 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 301560us
W20260709 12:14:22.473023 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.634037 26128 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.731s	user 0.003s	sys 0.007s
W20260709 12:14:22.697853 24014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.011s	sys 0.018s
I20260709 12:14:22.698060 23320 pending_rounds.cc:85] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 4
I20260709 12:14:22.703800 23320 pending_rounds.cc:107] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:14:22.704842 23320 raft_consensus.cc:2889] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.698843 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:22.698419 23389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.008s	sys 0.007s
I20260709 12:14:22.480450 26498 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:22.676841 26462 raft_consensus.cc:2764] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:14:22.572484 26500 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:14:22.397006 26510 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:22.519889 26471 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:22.520911 26311 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:22.629050 26277 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.557682 26383 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.281630 26450 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.545926 26469 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.710471 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.654284 26479 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.676638 26363 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:22.571928 26211 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:22.567710 26490 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.249729 26387 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.716634 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1981) took 1735 ms. Trace:
I20260709 12:14:22.794864 26499 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.795064 23320 rpcz_store.cc:276] 0709 12:14:20.984396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.988362 (+  3966us) service_pool.cc:224] Handling call
0709 12:14:21.001850 (+ 13488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.958057 (+956207us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5800204ed0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4
0709 12:14:22.716364 (+758307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.716574 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31802240}
W20260709 12:14:22.717991 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.765632 26327 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.798111 26327 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.799022 23316 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:22.800801 26327 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.798146 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 736) took 1053 ms. Trace:
I20260709 12:14:22.802131 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2303) took 1273 ms. Trace:
I20260709 12:14:22.802477 24008 rpcz_store.cc:276] 0709 12:14:21.745154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.071783 (+326629us) service_pool.cc:224] Handling call
0709 12:14:22.379403 (+307620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.677217 (+297814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.797897 (+120680us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b540063e7d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:22.798067 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675372288}
I20260709 12:14:22.802686 23470 rpcz_store.cc:276] 0709 12:14:21.528482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.070057 (+541575us) service_pool.cc:224] Handling call
0709 12:14:22.084337 (+ 14280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.229937 (+145600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.801829 (+571892us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002319d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:22.802051 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235097344}
I20260709 12:14:22.573608 26509 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:22.337272 26506 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:22.601257 26447 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.336547 26276 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.376577 26448 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.697088 26434 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:22.828048 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1894) took 1787 ms. Trace:
I20260709 12:14:22.767292 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:22.286453 26501 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.783875 26521 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:22.772593 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62073us
I20260709 12:14:22.836105 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:22.836745 23393 raft_consensus.cc:606] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:22.837123 23393 raft_consensus.cc:614] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:22.776280 23980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.011s	sys 0.027s
W20260709 12:14:22.779299 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.781551 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.768865 26471 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:22.628129 26433 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.766575 26311 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:22.822001 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.024s	sys 0.024s
I20260709 12:14:22.784106 26440 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.781822 26498 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:22.708213 26504 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:22.828599 24016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.029s	sys 0.059s
I20260709 12:14:22.780560 26315 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:22.850790 24016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.029s	sys 0.060s
I20260709 12:14:22.519062 26456 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.707933 26128 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.867540 26128 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.841756 26327 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:22.868258 26327 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.868458 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:22.868701 26327 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:22.869097 23395 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:22.854156 26502 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.869935 26486 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:22.870186 23396 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:14:22.852144 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 1.02294e+06us
I20260709 12:14:22.874639 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.770498 26429 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.769218 26520 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:22.706938 26503 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:22.837944 23980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.011s	sys 0.027s
W20260709 12:14:22.884039 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 720538us
I20260709 12:14:22.828645 23469 rpcz_store.cc:276] 0709 12:14:21.041073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.042445 (+  1372us) service_pool.cc:224] Handling call
0709 12:14:21.233587 (+191142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.234025 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.234081 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:21.601136 (+367055us) log.cc:844] Serialized 26 byte log entry
0709 12:14:21.952768 (+351632us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:21.952824 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.952931 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.827719 (+874788us) raft_consensus.cc:1685] finished
0709 12:14:22.827747 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.827958 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29356,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13562,"thread_start_us":155563,"threads_started":1,"wal-append.queue_time_us":448805}
I20260709 12:14:22.839576 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:22.842898 23316 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:22.794865 26387 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.891193 23390 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:22.870456 26486 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.893128 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 827) took 1226 ms. Trace:
I20260709 12:14:22.848507 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:22.850116 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.827580 26434 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.917485 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2276) took 1013 ms. Trace:
I20260709 12:14:22.918032 23389 rpcz_store.cc:276] 0709 12:14:21.903896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.904061 (+   165us) service_pool.cc:224] Handling call
0709 12:14:21.907916 (+  3855us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.908358 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:22.706259 (+797901us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.840284 (+134025us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:22.852164 (+ 11880us) log.cc:844] Serialized 26 byte log entry
0709 12:14:22.852320 (+   156us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:22.852355 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.852420 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.917242 (+ 64822us) raft_consensus.cc:1685] finished
0709 12:14:22.917269 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.917404 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7026,"prepare.queue_time_us":638347,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":47982,"raft.queue_time_us":819,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":16215,"thread_start_us":855106,"threads_started":2,"wal-append.queue_time_us":163973}
I20260709 12:14:22.920053 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:22.853081 26440 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.812836 26506 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.897768 24015 rpcz_store.cc:276] 0709 12:14:21.666865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.029193 (+362328us) service_pool.cc:224] Handling call
0709 12:14:22.047970 (+ 18777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.158276 (+110306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.158324 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.158382 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:22.158433 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.892926 (+734493us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540074e854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:22.892967 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.893075 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252390144}
I20260709 12:14:22.922739 26506 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:22.923008 26355 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.004s
I20260709 12:14:22.841924 26471 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:22.843685 26489 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.924329 26327 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.863157 26504 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.864689 26448 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.865466 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 583) took 1431 ms. Trace:
I20260709 12:14:22.840965 26435 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.860857 26456 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.927078 24016 rpcz_store.cc:276] 0709 12:14:21.434154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.951729 (+517575us) service_pool.cc:224] Handling call
0709 12:14:21.970890 (+ 19161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.971157 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:22.204695 (+233538us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.865131 (+660436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.865184 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:22.865229 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.865291 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.865395 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":113455,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":14567,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":827031,"threads_started":2,"wal-append.queue_time_us":686606}
I20260709 12:14:22.928814 26456 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.882897 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:22.930655 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.931258 23396 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:22.931943 26327 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.932260 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.845049 23468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:14:22.933810 23468 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:22.934172 23468 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:22.936066 26440 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.887208 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 1963) took 2357 ms. Trace:
I20260709 12:14:22.889561 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.889703 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 822) took 2098 ms. Trace:
W20260709 12:14:22.889778 26451 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 12:14:22.892311 23391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1557) took 2012 ms (client timeout 1694 ms). Trace:
I20260709 12:14:22.884742 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:22.791751 26363 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:22.845078 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.024s	sys 0.025s
W20260709 12:14:22.945478 23391 rpcz_store.cc:269] 0709 12:14:20.879633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.879981 (+   348us) service_pool.cc:224] Handling call
0709 12:14:22.892079 (+2012098us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002c8b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:22.892207 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666140416}
I20260709 12:14:22.893157 23390 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:14:22.899524 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60035us
I20260709 12:14:22.903311 26501 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.909982 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 581) took 1591 ms. Trace:
I20260709 12:14:22.911145 26498 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:22.939685 24014 rpcz_store.cc:276] 0709 12:14:20.791293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.811852 (+ 20559us) service_pool.cc:224] Handling call
0709 12:14:20.840124 (+ 28272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:21.953030 (+1112906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:21.953059 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:21.969001 (+ 15942us) log.cc:844] Serialized 26 byte log entry
0709 12:14:22.702508 (+733507us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:22.714872 (+ 12364us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.766946 (+ 52074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.767053 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.889485 (+122432us) raft_consensus.cc:1685] finished
0709 12:14:22.889504 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.889642 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83141,"prepare.queue_time_us":13664,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":86972,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":24998,"spinlock_wait_cycles":21456256,"thread_start_us":1745600,"threads_started":2,"wal-append.queue_time_us":863587}
I20260709 12:14:22.911752 26484 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.976016 23463 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:22.976310 26484 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:22.977190 26529 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:22.917590 26486 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:22.854079 26360 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:22.854103 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.981563 24008 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:22.929190 26458 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.885919 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.881086 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.983219 23393 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:22.970935 23468 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:22.933315 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:22.939103 26429 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:22.939229 23395 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:22.939801 23392 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:22.939507 23316 rpcz_store.cc:276] 0709 12:14:20.528732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.529650 (+   918us) service_pool.cc:224] Handling call
0709 12:14:20.541106 (+ 11456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.038134 (+497028us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b540019ce20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:14:22.798733 (+1760599us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b58005810d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4
0709 12:14:22.886059 (+ 87326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.886277 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623944320}
I20260709 12:14:22.973531 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:22.969517 26532 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:14:22.971555 24010 rpcz_store.cc:276] 0709 12:14:21.319052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.943264 (+624212us) service_pool.cc:224] Handling call
0709 12:14:22.909811 (+966547us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b540011d5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:22.909923 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137225088}
I20260709 12:14:22.973790 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:22.977890 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:22.885255 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:22.919127 26434 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:22.979475 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94820.1us
I20260709 12:14:23.004001 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:23.004976 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 119767us
I20260709 12:14:22.923480 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:22.985483 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:22.936805 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:22.995186 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2278) took 1059 ms. Trace:
I20260709 12:14:23.006309 23388 rpcz_store.cc:276] 0709 12:14:21.935309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.935554 (+   245us) service_pool.cc:224] Handling call
0709 12:14:22.994992 (+1059438us) spinlock_profiling.cc:243] Waited 64.1 ms on lock 0x7b54001211d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:22.995115 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147414912}
I20260709 12:14:22.969884 26504 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:23.009619 23389 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:23.009794 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 86384.6us
I20260709 12:14:22.981877 25950 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:23.010072 23389 raft_consensus.cc:614] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:23.004649 23390 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:22.982693 26435 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:23.007958 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.013887 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.014434 24014 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.016496 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.017493 24014 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:14:23.022588 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.011744 26501 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:23.025377 26501 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:23.026392 26501 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.032752 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.034848 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.035992 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.035569 23312 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:22.989293 26503 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:23.037992 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.013626 26435 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.039088 23320 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:23.054203 23315 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:23.040917 26433 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:23.041563 26277 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:23.037945 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:23.042745 23318 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:23.044622 26172 raft_consensus.cc:993] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:23.048002 26471 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.016821 26504 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.050496 26363 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:23.051010 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.983243 26529 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.036123 25950 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.057412 26435 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.061075 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.061995 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:23.063937 26501 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.069965 23469 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.070616 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:23.071373 23465 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.003695 26509 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:14:23.075809 24013 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:23.028954 26448 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:23.020795 26532 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:23.067312 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.077733 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.067508 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.075253 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.064838 26502 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:23.081132 23465 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:23.082639 23470 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:23.084250 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.084626 23462 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:23.085039 24010 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:23.069139 26433 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:23.102521 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:23.069357 26172 raft_consensus.cc:1081] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:23.102411 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:23.105301 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.103403 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.106591 24009 raft_consensus.cc:2410] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:23.109011 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.074733 26363 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:23.115836 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.116981 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:23.137672 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.140216 24016 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:23.141180 23469 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:23.142379 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.077472 26448 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.082931 26532 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.082417 26502 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:23.151274 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.155313 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.160554 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.162230 23469 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:23.163305 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.164198 23469 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:23.164739 23467 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2313) took 1537 ms (client timeout 1866 ms). Trace:
W20260709 12:14:23.165160 23467 rpcz_store.cc:269] 0709 12:14:21.627305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.954212 (+326907us) service_pool.cc:224] Handling call
0709 12:14:23.164536 (+1210324us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54001144d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:23.164663 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260509952}
I20260709 12:14:23.120843 26363 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.166769 23461 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 5. Ops: [3.6-3.6]
I20260709 12:14:23.169118 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.170188 23462 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:23.081768 26128 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:14:23.081924 26173 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:14:23.138731 26434 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:23.144981 26446 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:23.179857 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.149128 26532 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.187712 24015 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:23.150219 26448 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.183769 26327 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.185631 26173 consensus_queue.cc:1243] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:23.188190 26434 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.185848 26458 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.200006 26434 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.200505 26502 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.201687 25950 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.202324 26433 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.191386 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.205686 23469 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:14:23.186929 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:23.206422 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 490108us
I20260709 12:14:23.178856 26363 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:23.193778 26327 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.208112 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.212393 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:23.213649 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 150265us
I20260709 12:14:23.217244 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:23.216464 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.220785 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.217052 23462 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.225101 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.236416 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.237574 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.239086 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.243194 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.219669 26456 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:23.246299 26456 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:23.246762 26456 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.244590 26276 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:14:23.245334 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.224552 26506 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:23.224201 26447 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:23.262938 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.235350 26128 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:23.272125 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 192780us
I20260709 12:14:23.245867 26479 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:23.272838 26360 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:23.274611 26276 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:23.284415 26529 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:23.261121 26447 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.285035 26276 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.285725 26447 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:23.275640 26550 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 12:14:23.278553 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.245482 26327 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.256916 26458 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:23.272876 26450 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:23.282958 26544 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:14:23.256000 26549 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:23.281718 26471 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.260692 26506 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:23.296152 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.301479 23287 leader_election.cc:400] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9.
I20260709 12:14:23.321568 23287 leader_election.cc:403] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:23.310346 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.260406 26172 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:23.327179 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45805.2us
I20260709 12:14:23.329758 26450 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9.
I20260709 12:14:23.330796 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.331772 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.332751 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.333933 26450 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:23.334499 26450 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:23.334913 26450 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.336439 26532 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.337661 26327 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:23.338142 26327 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:23.338506 26327 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.339113 26532 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:23.317792 26485 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:14:23.325978 26211 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:14:23.347028 26548 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.347589 26548 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.349164 26435 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.349917 26446 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.350687 26446 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.351557 26435 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.359897 26450 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.301249 26491 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.365866 26491 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.368290 26491 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.368291 26450 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.373140 26450 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:23.371646 23462 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:23.376943 26311 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.377360 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:23.378051 26311 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.380964 23462 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:23.323043 26506 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:23.383044 26327 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.384238 23464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:14:23.384773 23464 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:23.385074 26327 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.386947 23466 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:23.392916 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 745) took 1322 ms. Trace:
W20260709 12:14:23.395556 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.403839 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.406924 26503 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:23.407495 26503 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:23.412406 26521 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.414005 26172 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:23.415205 26504 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.415021 26172 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.422854 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.405228 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.385177 23464 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.424688 24013 rpcz_store.cc:276] 0709 12:14:22.070398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.519188 (+448790us) service_pool.cc:224] Handling call
0709 12:14:22.671124 (+151936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.076211 (+405087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.392624 (+316413us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54006cb954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:23.392839 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930238208}
I20260709 12:14:23.432472 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:23.408587 26503 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.388686 26456 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:23.439002 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.460753 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:23.461521 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.462265 23316 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.446996 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.462961 23317 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.431773 26486 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:23.465296 23391 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:23.468290 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:23.439092 26532 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:23.477043 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 144346us
I20260709 12:14:23.482023 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.483114 23313 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.483985 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:23.485045 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.486630 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:23.489331 23316 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:23.435218 26435 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:14:23.490343 23319 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:23.489362 23388 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:23.441731 26471 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 12:14:23.520906 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 200722us
I20260709 12:14:23.468487 26503 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.468389 26446 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.525233 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.526926 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:23.527467 23319 raft_consensus.cc:2410] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:23.441574 26491 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:14:23.525362 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.526407 23392 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:23.523018 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.532874 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:23.533298 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:23.438484 26433 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 12:14:23.536754 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.537724 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:23.537830 26355 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:23.538246 26532 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:23.540952 26456 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.524868 26327 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:23.537866 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:23.546939 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:23.568497 23316 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:23.549986 23317 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:23.557119 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:23.559499 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.571362 23319 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.549583 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.439811 26447 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.577389 23313 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:23.563611 23464 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.533174 26491 consensus_queue.cc:1243] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:14:23.574960 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.574946 23317 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:23.523998 26448 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.551301 26446 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:23.590207 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.552321 26485 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.591598 23390 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:23.593551 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.594705 23317 raft_consensus.cc:1217] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-8.7]   Dedup: 4.5->[6.6-8.7]
I20260709 12:14:23.595187 23317 pending_rounds.cc:85] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:14:23.595423 23317 pending_rounds.cc:107] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:14:23.567384 26327 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:23.568928 26355 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:23.570741 25950 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.595702 23317 raft_consensus.cc:2889] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.596721 23464 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:23.600793 23319 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:23.570611 26564 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:23.589814 26450 raft_consensus.cc:2764] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:14:23.601549 26501 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.004s
I20260709 12:14:23.624610 26532 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:23.626446 26487 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.626892 26563 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:23.561116 26524 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:14:23.648298 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.646504 24009 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:23.615684 26363 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.616627 25950 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.655222 26549 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:23.655741 26363 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.608990 26327 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.658174 26448 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.630139 26532 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:23.659242 26532 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.596621 26446 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.663092 26446 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:23.652851 26355 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.666360 26479 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:23.667143 23313 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.650447 26501 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:14:23.656087 25950 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.668160 26501 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:23.668646 26501 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:23.660396 26506 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.594094 26521 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:23.669631 26501 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.670782 26501 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:23.670910 26506 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.672073 23316 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 3. Current term is 4. Ops: []
W20260709 12:14:23.646982 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48738.6us
W20260709 12:14:23.674182 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 124681us
W20260709 12:14:23.738233 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 144780us
W20260709 12:14:23.738401 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.741539 23464 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:23.738794 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:23.745407 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:23.751431 26532 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.751952 26363 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:23.753718 26450 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:23.755700 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.767678 23313 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:23.770493 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:23.770469 24014 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:23.771164 24015 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:23.771409 24013 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:23.770087 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.779887 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:23.756364 26471 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:23.784638 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:23.786163 23462 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:23.786289 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.734709 26506 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:23.789163 26506 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:23.789690 26506 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:23.790017 26506 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:23.772204 26450 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:23.790500 26450 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.791260 26506 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.792896 26506 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:23.777969 26549 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.793813 26549 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.795949 26549 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:23.796645 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.787228 23320 raft_consensus.cc:2410] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:14:23.787990 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:23.799460 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.799942 23464 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:23.778226 26456 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.778425 26458 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.805578 23462 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:23.783121 26569 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.808126 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.783954 26471 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:23.772732 26363 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.819983 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.797006 26172 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:23.812505 26471 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.807368 26456 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:23.830353 23464 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:23.830016 26572 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:14:23.831333 26572 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:23.831734 26456 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.831802 26572 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:23.834353 26572 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.837962 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.839227 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.840252 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.841519 23318 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:23.845453 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.846848 23318 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:23.810256 26581 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:14:23.847887 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.853758 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.856087 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.856597 23318 raft_consensus.cc:2368] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5.
I20260709 12:14:23.853492 26564 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:23.864475 23318 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:14:23.866575 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.869817 26139 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.870308 26327 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.873618 26327 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:23.880424 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.885775 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:23.886579 23387 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:23.886924 23387 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:23.887821 23387 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:23.888978 23387 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:23.856964 26532 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.908022 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:23.884958 26548 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:23.908318 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.908712 23389 raft_consensus.cc:606] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:23.916814 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.917639 23314 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:14:23.927852 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:23.933954 26585 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:14:23.940109 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.941118 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:23.941915 23314 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:23.947827 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.948298 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.004544 24014 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:23.970371 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.005208 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.944573 26450 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.006466 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.007105 23312 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:24.007550 26450 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:24.007900 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.970907 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:23.946537 26139 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:23.974924 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:24.009109 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:23.949749 24016 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:14:24.012557 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.013651 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.014140 23319 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.014688 23319 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:24.015349 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.015618 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.016153 23314 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8.
I20260709 12:14:24.016310 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.016973 23319 raft_consensus.cc:2368] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8.
I20260709 12:14:24.017861 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.018905 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.021683 23315 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:24.021797 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.023108 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.023795 23320 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:24.025933 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:24.026727 24016 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:23.940346 26363 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:23.970234 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:24.030455 23318 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:24.036269 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.041766 23315 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:24.043781 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.049724 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.052081 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:24.038411 26506 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.057315 26506 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.060842 26506 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:24.062904 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:24.082945 23287 leader_election.cc:400] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9.
I20260709 12:14:24.063218 26503 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:24.083628 26503 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.084714 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:24.084961 23468 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:24.085444 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36038 (request call id 1905) took 1240 ms. Trace:
I20260709 12:14:24.085718 26503 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:24.032442 26504 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:14:24.086951 26506 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:24.087393 26506 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.087520 26504 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:24.087764 26506 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:24.087994 26504 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:24.088996 26504 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.082994 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:24.091696 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.091853 23319 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:24.092939 23313 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
W20260709 12:14:24.093542 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.094133 23313 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:24.094764 23391 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:24.095156 23313 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:24.095116 26561 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:24.095978 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:24.096504 23313 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:24.063901 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:24.064185 26489 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:14:24.064337 23312 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:14:24.067296 26443 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.097404 23465 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:24.098613 26443 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.097715 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:24.099841 24011 raft_consensus.cc:2376] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8. Re-sending same reply.
W20260709 12:14:24.100301 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 752) took 1251 ms (client timeout 1511 ms). Trace:
W20260709 12:14:24.100620 24011 rpcz_store.cc:269] 0709 12:14:22.848758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.848879 (+   121us) service_pool.cc:224] Handling call
0709 12:14:24.097583 (+1248704us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:24.100249 (+  2666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22403456}
I20260709 12:14:24.097745 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.085758 23468 rpcz_store.cc:276] 0709 12:14:22.844815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.844936 (+   121us) service_pool.cc:224] Handling call
0709 12:14:24.084857 (+1239921us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4c001ec300. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7940a20 00007ff0a7951a29 00007ff0a795213f 00007ff0a79162ab 00007ff0a7943f3c 00007ff0a79443fe 00007ff0a7bd188b 00007ff0a95d0495 00007ff0a962130f 00007ff0a966ba55 00007ff0ae63bcd0
0709 12:14:24.085339 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":277,"spinlock_wait_cycles":26317568}
I20260709 12:14:24.098039 26489 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.103991 23462 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:14:24.103803 26489 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.104888 26581 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:24.105119 26489 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:24.106305 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2301) took 2578 ms. Trace:
I20260709 12:14:24.106736 23461 rpcz_store.cc:276] 0709 12:14:21.527972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.063995 (+536023us) service_pool.cc:224] Handling call
0709 12:14:23.166455 (+1102460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.173237 (+  6782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.106110 (+932873us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001144d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:24.106246 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503176448}
I20260709 12:14:24.108192 26572 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:24.009466 26139 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.075731 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 750) took 1227 ms. Trace:
I20260709 12:14:24.111728 24008 rpcz_store.cc:276] 0709 12:14:22.848104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.848259 (+   155us) service_pool.cc:224] Handling call
0709 12:14:24.075548 (+1227289us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b54007f0154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:24.075669 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71173504}
I20260709 12:14:24.079349 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:24.080294 23464 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 2. Current term is 3. Ops: []
I20260709 12:14:24.112851 23470 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:24.115206 23434 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.511s (SENT)
I20260709 12:14:24.115595 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 555) took 1134 ms. Trace:
I20260709 12:14:24.115944 23463 rpcz_store.cc:276] 0709 12:14:22.981366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.981531 (+   165us) service_pool.cc:224] Handling call
0709 12:14:23.173532 (+192001us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.173998 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.174040 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.191916 (+ 17876us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.295419 (+103503us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:23.295672 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.297301 (+  1629us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:23.338579 (+ 41278us) raft_consensus.cc:1685] finished
0709 12:14:23.338601 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.115369 (+776768us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54001144d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:24.115544 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4359,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7922,"spinlock_wait_cycles":386240768,"thread_start_us":75591,"threads_started":1,"wal-append.queue_time_us":117330}
I20260709 12:14:24.119560 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 839) took 1011 ms. Trace:
I20260709 12:14:24.081683 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:24.071841 26562 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.530s	user 0.003s	sys 0.000s
I20260709 12:14:24.042534 26471 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.119959 24009 rpcz_store.cc:276] 0709 12:14:23.108199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.108788 (+   589us) service_pool.cc:224] Handling call
0709 12:14:24.119394 (+1010606us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400780d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:24.119504 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234454400}
I20260709 12:14:24.122159 26575 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:24.095120 23391 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.123853 26471 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:24.121671 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 558) took 1060 ms. Trace:
I20260709 12:14:24.124550 23467 rpcz_store.cc:276] 0709 12:14:23.060983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.191356 (+130373us) service_pool.cc:224] Handling call
0709 12:14:23.209578 (+ 18222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.216621 (+  7043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.216681 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.222475 (+  5794us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.422211 (+199736us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:23.433933 (+ 11722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.434046 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.071227 (+637181us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:24.121356 (+ 50129us) raft_consensus.cc:1685] finished
0709 12:14:24.121377 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.121601 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":7744,"spinlock_wait_cycles":5235072,"thread_start_us":172150,"threads_started":1,"wal-append.queue_time_us":291908}
I20260709 12:14:24.123536 23391 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.126509 26172 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:24.126910 26172 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.098258 23315 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:24.127745 26172 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:24.130235 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.133415 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.134562 26549 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:24.135052 26549 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.136257 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.137068 26276 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:24.136940 26535 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:24.137545 26276 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.137974 26276 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:24.141228 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:24.142105 24016 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:24.147241 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:24.135934 26549 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.149032 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:24.126550 23391 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:24.152706 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 135327us
W20260709 12:14:24.163318 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.152323 26447 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:24.169976 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.173498 23467 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:14:24.180135 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.180444 23387 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:24.213593 26139 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:24.217571 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47922.7us
I20260709 12:14:24.220436 26443 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:24.218696 23319 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:14:24.225255 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.219424 26504 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:24.413848 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.463831 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 377176us
I20260709 12:14:24.427776 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:24.437749 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:24.438202 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 217645us
I20260709 12:14:24.464846 23316 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:24.272850 23318 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:24.382016 23312 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
W20260709 12:14:24.305176 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.278009 23391 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
W20260709 12:14:24.465214 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.474022 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.269639 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.813s (SENT) [suppressed 19 similar messages]
I20260709 12:14:24.260304 23463 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:24.478955 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.375808 23315 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:24.480516 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.479859 26535 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:24.482204 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 212616us
I20260709 12:14:24.229532 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.468447 23312 pending_rounds.cc:85] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:14:24.491370 23312 pending_rounds.cc:107] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:14:24.175241 26503 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.484761 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:24.495324 23470 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:24.487932 23315 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:24.228890 26172 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:24.492475 23318 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:24.203697 26387 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:24.509413 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:24.510066 23470 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:24.491840 23312 raft_consensus.cc:2889] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:24.494136 26559 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:14:24.496129 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:24.502803 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:24.512456 23462 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:24.513813 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:24.503639 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 274207us
I20260709 12:14:24.464699 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.473317 26443 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:14:24.521188 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.515375 26139 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:24.522917 26139 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:24.524173 26139 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.532138 26520 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:24.532663 26520 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.533032 26520 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:24.533983 26555 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:24.534569 26555 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.534996 26555 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.535221 26443 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:24.535668 26443 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:24.536531 26443 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.538064 26575 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:24.539721 26276 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.534026 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:24.540526 26470 consensus_queue.cc:1175] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260709 12:14:24.541852 26470 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:24.542232 26470 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.542565 26470 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:24.174682 26509 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.328s
I20260709 12:14:24.545770 23461 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:14:24.547417 23436 leader_election.cc:336] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.728s (SENT)
I20260709 12:14:24.550377 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:24.482745 26574 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:14:24.549098 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:24.561620 24016 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:14:24.561605 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:24.546608 26535 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:24.575547 23470 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:24.502352 26503 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:24.579035 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.554301 26588 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:24.566098 26560 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:24.520361 26506 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.582248 26443 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:24.579286 26509 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:24.583114 26535 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.606419 23320 raft_consensus.cc:1217] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 12:14:24.607102 23320 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: [4.6-10.9]
I20260709 12:14:24.603363 23316 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:24.605587 23312 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:14:24.594540 26486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.042s	sys 0.072s
I20260709 12:14:24.591413 26560 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:24.609997 26486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.066s	user 0.042s	sys 0.072s
I20260709 12:14:24.608814 26276 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:24.625490 26486 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:24.641763 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 145294us
W20260709 12:14:24.648736 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.656957 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:24.677109 26532 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:24.678349 26532 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:24.678826 26532 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:24.680266 26532 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.682135 26532 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:24.699558 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:24.690076 26501 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:14:24.713330 23434 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.225s (SENT)
I20260709 12:14:24.714179 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:24.715657 26588 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:24.671360 26211 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:14:24.686817 26574 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:14:24.710297 26503 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:24.743294 26489 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.011s
I20260709 12:14:24.744140 26489 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:24.669381 26560 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.723784 26501 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:24.749948 26560 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:24.750586 26501 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:24.753180 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.754310 26574 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:24.754977 26574 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:24.756646 26574 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.769129 26506 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:24.770140 26506 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:24.707394 26276 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:14:24.789481 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89917.5us
I20260709 12:14:24.688557 26535 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:24.791391 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 229884us
I20260709 12:14:24.744942 26520 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.698473 26486 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.785641 26471 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:14:24.797974 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:24.804098 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 127475us
W20260709 12:14:24.799489 26591 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:14:24.802497 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 337868us
W20260709 12:14:24.783428 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 291114us
I20260709 12:14:24.808601 26447 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:24.807129 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:24.811923 26139 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.812448 26139 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.816211 26506 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:24.816260 26575 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:24.816679 26506 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:24.818207 26506 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:24.822064 26520 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:24.822418 26462 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:24.822836 26575 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:24.671280 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:24.816648 26470 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.826292 26470 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:24.828369 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:24.829066 23463 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:24.830862 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.834107 26139 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:24.835762 23216 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 326) took 16227 ms (client timeout 15000 ms). Trace:
I20260709 12:14:24.835875 26555 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.844750 26555 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:24.847427 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:24.839869 23216 rpcz_store.cc:269] 0709 12:14:08.608418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.608787 (+   369us) service_pool.cc:224] Handling call
0709 12:14:24.835664 (+16226877us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:23.160319 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:23.160975 (+   656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:23.161025 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:14:23.161047 (+    22us) tablet.cc:662] Decoding operations
0709 12:14:23.162299 (+  1252us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:23.162406 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:23.162439 (+    33us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:14:23.163420 (+   981us) tablet.cc:701] Row locks acquired
0709 12:14:23.163469 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:14:23.163704 (+   235us) write_op.cc:270] Start()
0709 12:14:23.163844 (+   140us) write_op.cc:276] Timestamp: P: 1783599263163594 usec, L: 0
0709 12:14:23.163862 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:14:23.164523 (+   661us) log.cc:844] Serialized 4054 byte log entry
0709 12:14:23.589485 (+424962us) op_driver.cc:464] REPLICATION: finished
0709 12:14:23.825650 (+236165us) write_op.cc:301] APPLY: starting
0709 12:14:23.828103 (+  2453us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:23.840922 (+ 12819us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:23.841040 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:23.858362 (+ 17322us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:23.858684 (+   322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:14:23.858740 (+    56us) write_op.cc:312] APPLY: finished
0709 12:14:23.859018 (+   278us) log.cc:844] Serialized 77 byte log entry
0709 12:14:23.859772 (+   754us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:23.859866 (+    94us) write_op.cc:454] Released schema lock
0709 12:14:23.859899 (+    33us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:23.860285 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:23.860315 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1765023,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":220914,"apply.run_cpu_time_us":3818,"apply.run_wall_time_us":37859,"num_ops":10,"prepare.queue_time_us":113559,"prepare.run_cpu_time_us":5854,"prepare.run_wall_time_us":21382,"raft.queue_time_us":244439,"raft.run_cpu_time_us":13404,"raft.run_wall_time_us":146218,"replication_time_us":425431,"spinlock_wait_cycles":21248,"thread_start_us":377310,"threads_started":4,"wal-append.queue_time_us":17033}]]}
I20260709 12:14:24.850787 26506 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.844888 23463 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:24.846459 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:24.855933 23466 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.857867 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.867619 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:24.870880 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:24.876250 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.877763 23392 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:24.877766 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.863673 26429 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.884383 26506 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:24.867113 26552 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:24.889494 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 81693us
I20260709 12:14:24.875986 26139 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:24.895179 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.903148 23466 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:24.908034 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:24.908311 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.908851 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.806974 26498 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:14:24.878527 25950 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:24.911334 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:24.866838 26509 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
W20260709 12:14:24.904270 26572 consensus_meta.cc:287] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.788s	user 0.016s	sys 0.000s
I20260709 12:14:24.880615 26548 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:24.905575 26506 raft_consensus.cc:2764] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:24.922608 26490 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:24.923738 26509 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:24.925482 26598 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:24.926704 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:24.912575 26139 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:24.940662 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:24.895401 26429 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:24.918309 26552 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:24.907904 26597 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:14:24.960364 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.928567 25950 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:24.968369 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.936463 26509 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:24.948309 26490 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:24.972795 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:24.972774 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:24.977397 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.976393 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:24.952564 26572 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:24.980886 25950 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.982080 26552 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.960441 26462 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:24.985487 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:24.989987 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.993305 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:24.942500 26139 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:24.929867 26613 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:14:24.991518 23395 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:24.995108 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.996007 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.931671 26548 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:24.972398 26449 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.002794 26533 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:25.002895 26509 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.005640 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:25.010383 23396 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:25.011270 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:25.011651 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:25.011943 23393 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.012794 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:25.018066 26490 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.023104 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:25.025003 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:25.031389 23393 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:24.987725 26600 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:25.040357 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:24.997684 26572 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:25.053242 26470 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:25.055992 26572 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.025485 26533 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:25.060385 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.049118 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:25.038868 26555 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:25.040635 26486 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.061378 26561 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:14:25.033293 26498 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:25.046924 26471 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:25.046633 23395 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:25.060602 26128 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:14:25.138775 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.126122 26560 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.034337 26449 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.065138 26509 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:25.140861 23466 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:25.142256 26620 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.144902 23468 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:14:25.145830 26501 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:25.149554 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.152129 23393 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:14:25.136924 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 87937.9us
I20260709 12:14:25.062088 26600 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:25.156666 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:25.193382 23463 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.158218 23395 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:14:25.168628 26502 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:25.150530 26486 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:25.151189 26595 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:14:25.139396 26597 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.126156 26506 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:25.066535 26548 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:25.219213 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.217113 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 368442us
I20260709 12:14:25.203225 23467 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:14:25.205339 24012 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:14:25.173547 26533 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:25.210469 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.228509 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:25.236446 23465 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:25.236616 23395 raft_consensus.cc:1217] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Deduplicated request from leader. Original: 1.4->[4.5-7.6]   Dedup: 7.6->[]
I20260709 12:14:25.237403 23395 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [4.5-7.6]
I20260709 12:14:25.215209 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 706) took 1068 ms. Trace:
I20260709 12:14:25.237993 23320 rpcz_store.cc:276] 0709 12:14:24.146593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.167977 (+ 21384us) service_pool.cc:224] Handling call
0709 12:14:24.229572 (+ 61595us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:14:24.611852 (+382280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.214811 (+602959us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:25.215125 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":866006400}
I20260709 12:14:25.193102 26470 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.201943 26487 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.244990 24009 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:25.278360 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:25.176618 26498 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:25.304270 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.178350 26560 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.218202 26504 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:25.323711 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:25.328195 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:25.340443 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:25.340372 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.338985 23463 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:25.258790 26533 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:25.226898 26363 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 12:14:25.281607 26555 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.227288 26506 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.383903 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:25.375193 24009 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:25.375849 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:25.377646 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.536s (SENT)
I20260709 12:14:25.386214 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:25.386713 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:25.229161 26561 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:14:25.380872 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:25.387821 23314 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:25.374974 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.204260 26620 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.323191 26429 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:25.322714 26447 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:25.224229 26597 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.402875 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:25.403798 24013 raft_consensus.cc:1217] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[4.5-7.6]   Dedup: 7.6->[]
I20260709 12:14:25.404604 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.430004 24014 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:25.397063 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:25.431501 23465 raft_consensus.cc:2368] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
I20260709 12:14:25.435667 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:25.426538 26429 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.443641 26429 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.444150 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.445698 26561 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:25.446040 26449 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:25.450995 26311 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:25.405308 26621 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:14:25.409873 26600 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.302268 26487 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.454648 26498 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:25.455375 26487 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:25.282052 26470 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.456621 24013 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [4.5-7.6]
W20260709 12:14:25.456911 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 81145.3us
W20260709 12:14:25.457721 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.457216 23465 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:14:25.462040 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84416.6us
W20260709 12:14:25.462916 23434 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.553s (SENT)
I20260709 12:14:25.463392 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:25.459458 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70090.4us
I20260709 12:14:25.382827 26612 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:25.482029 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:25.482685 23468 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:25.477015 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:25.468008 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9248074849999739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 12:14:25.484822 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9167074849999608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 12:14:25.485145 23287 connection.cc:256] RPC call timeout handler was delayed by -2.8654074850001052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
W20260709 12:14:25.485589 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7565074849999291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 12:14:25.485891 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7214074850000998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
W20260709 12:14:25.486261 23287 connection.cc:256] RPC call timeout handler was delayed by -2.6683074850000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 12:14:25.486615 23287 connection.cc:256] RPC call timeout handler was delayed by -2.6674074850000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:14:25.486960 23287 connection.cc:256] RPC call timeout handler was delayed by -2.3524074849999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260709 12:14:25.487314 23287 connection.cc:256] RPC call timeout handler was delayed by -2.32900748499992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 12:14:25.487656 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9600074850000055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26530000000000004s for a response
W20260709 12:14:25.488003 23287 connection.cc:256] RPC call timeout handler was delayed by -1.7989074849999724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2832s for a response
I20260709 12:14:25.489349 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:25.485162 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:25.453164 26462 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:25.484314 23320 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:25.530092 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.514317 25950 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:25.541812 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:25.542701 24009 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:25.542824 23320 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:25.373164 26504 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.446143 26561 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:25.548204 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:25.535082 26139 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.001s
I20260709 12:14:25.476540 26548 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:25.574335 23314 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:14:25.557717 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 68473us
I20260709 12:14:25.391681 26555 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.475665 26471 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:25.565150 24016 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:14:25.410231 26276 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.141s
I20260709 12:14:25.578228 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:25.581808 23317 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.007s
W20260709 12:14:25.594980 23317 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.008s
W20260709 12:14:25.602407 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 177108us
I20260709 12:14:25.457206 26447 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.605420 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:25.619886 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.574641 25950 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:14:25.702051 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 96180.3us
I20260709 12:14:25.728860 26612 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:25.730289 26612 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.737847 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2009) took 2170 ms. Trace:
W20260709 12:14:25.742827 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 161833us
I20260709 12:14:25.573937 26139 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.738370 23317 rpcz_store.cc:276] 0709 12:14:23.567832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.593928 (+ 26096us) service_pool.cc:224] Handling call
0709 12:14:23.594519 (+   591us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:23.616401 (+ 21882us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:24.027860 (+411459us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b18002b7bc0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0ad06c3c7 00007ff0a967e9a7 00007ff0a968e0bf 00007ff0a968e038
0709 12:14:24.382546 (+354686us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.881731 (+499185us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:24.883305 (+  1574us) log.cc:844] Serialized 52 byte log entry
0709 12:14:24.890067 (+  6762us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:25.062686 (+172619us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.595545 (+532859us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.734369 (+138824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.734602 (+   233us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:25.737541 (+  2939us) raft_consensus.cc:1685] finished
0709 12:14:25.737565 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.737773 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17495,"prepare.queue_time_us":47212,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":62002,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":20998,"spinlock_wait_cycles":7748352,"thread_start_us":1386006,"threads_started":5,"wal-append.queue_time_us":324444}
W20260709 12:14:25.749111 26595 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:14:25.638753 26429 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:25.591558 26355 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.144s
W20260709 12:14:25.795594 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 217453us
I20260709 12:14:25.804742 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:25.781629 26593 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:14:25.820870 23466 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:25.828681 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:25.831990 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 330076us
W20260709 12:14:25.843014 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 98025.8us
I20260709 12:14:25.828145 26355 consensus_queue.cc:1243] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:25.853605 25950 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:25.915308 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86739.1us
I20260709 12:14:25.939108 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:25.939826 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.941582 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.942178 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.943058 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:25.943356 23320 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:25.943694 23314 raft_consensus.cc:2376] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3. Re-sending same reply.
I20260709 12:14:25.944427 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.944628 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.853403 26560 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:25.736032 26561 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.945978 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.946475 26561 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:25.947188 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.948269 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.942503 23317 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:25.950350 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.951489 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.952080 23317 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:25.952297 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:25.953148 23320 raft_consensus.cc:2376] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6. Re-sending same reply.
I20260709 12:14:25.954242 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:25.921864 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.955092 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.955993 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.949342 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.959244 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.957202 23320 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:25.962131 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.963949 26355 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:25.908862 26632 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
W20260709 12:14:25.933188 26625 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.310s	user 0.002s	sys 0.000s
W20260709 12:14:25.933480 26614 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.539s	user 0.002s	sys 0.000s
I20260709 12:14:25.962721 23317 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:25.967137 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:25.848876 26429 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:25.995924 23316 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.006s	sys 0.016s
I20260709 12:14:25.900714 26662 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:25.933588 26631 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 12:14:26.009123 23316 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.006s	sys 0.017s
I20260709 12:14:25.908380 26506 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.011699 23314 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:26.015442 26662 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.043229 26662 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.042655 26506 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.044106 26506 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:25.966611 26635 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:14:26.046152 26649 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:14:26.047456 26506 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.048055 26506 raft_consensus.cc:2764] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:26.048643 26450 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:26.049152 26450 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.050158 26642 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.050184 26450 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.050901 26642 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.051679 26501 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:26.052083 26501 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:26.009302 26355 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:26.073395 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 210139us
W20260709 12:14:26.077823 23462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.006s	sys 0.015s
W20260709 12:14:26.081336 23462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.006s	sys 0.015s
I20260709 12:14:26.053330 26642 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:26.090201 26628 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
W20260709 12:14:26.093035 26622 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
W20260709 12:14:26.064756 26600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.010s	sys 0.029s
W20260709 12:14:26.097709 26600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.010s	sys 0.029s
I20260709 12:14:26.097819 26139 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.099982 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 523) took 1139 ms. Trace:
I20260709 12:14:26.024374 26429 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:25.975215 25950 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.112599 23317 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:26.052965 26501 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.120551 23317 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:26.124449 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.023s	sys 0.036s
I20260709 12:14:26.066395 26597 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:26.128743 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.024s	sys 0.036s
I20260709 12:14:26.009590 26634 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:14:26.100461 23396 rpcz_store.cc:276] 0709 12:14:24.960744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.013694 (+ 52950us) service_pool.cc:224] Handling call
0709 12:14:25.014040 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:25.034144 (+ 20104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.034184 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:25.034906 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 12:14:25.367391 (+332485us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:25.367436 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.367511 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.099666 (+732155us) raft_consensus.cc:1685] finished
0709 12:14:26.099693 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.099882 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19662,"raft.queue_time_us":26789,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3170,"thread_start_us":300060,"threads_started":1,"wal-append.queue_time_us":433797}
I20260709 12:14:26.082166 26555 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.049147 26506 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 12:14:26.285148 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 799998us
I20260709 12:14:26.285003 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.287497 23314 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:26.104043 26620 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.103891 26447 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.335336 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:26.192528 26615 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 12:14:26.157397 26533 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.001s
W20260709 12:14:26.154654 26572 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.098s	user 0.012s	sys 0.000s
W20260709 12:14:26.193801 26502 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.988s	user 0.010s	sys 0.003s
I20260709 12:14:26.305693 26520 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.136871 26636 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.283653 26429 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:26.344558 26486 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.345471 25950 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.350380 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 851) took 1324 ms. Trace:
W20260709 12:14:26.350637 26509 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.165s	user 0.012s	sys 0.000s
I20260709 12:14:26.351666 26525 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:26.353953 26503 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.351770 23434 leader_election.cc:400] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8.
I20260709 12:14:26.379583 23434 leader_election.cc:403] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:26.144695 23317 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.364677 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.381057 23317 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:26.291210 26211 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.371309 24016 rpcz_store.cc:276] 0709 12:14:25.026025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.026727 (+   702us) service_pool.cc:224] Handling call
0709 12:14:25.057225 (+ 30498us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:25.579759 (+522534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.350068 (+770309us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b540044ff54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:26.350306 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167340288}
I20260709 12:14:26.155628 26664 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.301s
I20260709 12:14:26.389670 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.396209 26617 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.396823 26617 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.397871 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2038) took 1068 ms. Trace:
I20260709 12:14:26.398097 26676 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.398593 26676 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.398308 23316 rpcz_store.cc:276] 0709 12:14:25.329798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.332063 (+  2265us) service_pool.cc:224] Handling call
0709 12:14:25.332443 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.029003 (+696560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.029139 (+   136us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.043340 (+ 14201us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.208174 (+164834us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:26.208229 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.208326 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.397544 (+189218us) raft_consensus.cc:1685] finished
0709 12:14:26.397574 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.397804 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72041,"raft.queue_time_us":481709,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":34942,"thread_start_us":810094,"threads_started":2,"wal-append.queue_time_us":350882}
I20260709 12:14:26.400321 26555 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.400609 26658 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.401206 26658 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.401571 26490 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.402400 26490 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.405447 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.405889 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.407322 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1955) took 1212 ms. Trace:
I20260709 12:14:26.389761 26612 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:26.405306 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.411988 26662 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.415704 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.400871 26555 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.407742 23466 rpcz_store.cc:276] 0709 12:14:25.194547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.194677 (+   130us) service_pool.cc:224] Handling call
0709 12:14:25.194961 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.823132 (+628171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.407097 (+583965us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400230fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:26.407251 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438841216}
I20260709 12:14:26.379273 26525 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:26.419529 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.155359 26355 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.420737 24015 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:26.422832 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.368891 26572 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.429760 23317 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:26.370203 26520 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.430855 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.432369 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.391191 26502 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8.
I20260709 12:14:26.405012 26490 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.437388 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.437790 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.441143 24007 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:26.443534 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.421772 26588 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:26.445123 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.451105 24007 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:26.448040 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.445576 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.371521 26636 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.454593 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.433794 26503 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.389331 26211 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:26.461087 26630 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.710s	user 0.005s	sys 0.002s
I20260709 12:14:26.461684 26672 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:26.473768 26672 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.474491 26636 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.444486 26639 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.474975 26672 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.456087 26450 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:26.481938 26450 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.482445 26450 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:26.484643 26504 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:26.450125 25950 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:26.461889 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 724) took 1126 ms. Trace:
W20260709 12:14:26.486701 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.462093 26658 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.486456 23321 rpcz_store.cc:276] 0709 12:14:25.335510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.335617 (+   107us) service_pool.cc:224] Handling call
0709 12:14:25.335904 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.506547 (+170643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.506592 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.507155 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.832049 (+324894us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:25.832105 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.832210 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.461624 (+629414us) raft_consensus.cc:1685] finished
0709 12:14:26.461650 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.461824 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59792,"raft.queue_time_us":491768,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":32120,"thread_start_us":460885,"threads_started":2,"wal-append.queue_time_us":952841}
I20260709 12:14:26.462743 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 854) took 1397 ms. Trace:
I20260709 12:14:26.462638 26662 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.449262 26617 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.464874 26525 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.494503 25950 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.496865 26525 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:26.496915 25950 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.497887 26128 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:26.498262 26128 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:26.499102 26128 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.499982 25950 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.465782 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2031) took 1660 ms. Trace:
I20260709 12:14:26.500277 26128 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.458039 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.469679 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.466639 26502 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.503307 26662 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:26.492051 26600 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.503829 26662 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.504392 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 518) took 1548 ms. Trace:
I20260709 12:14:26.504273 26600 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.504714 26662 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.506601 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 856) took 1334 ms. Trace:
I20260709 12:14:26.500571 23312 rpcz_store.cc:276] 0709 12:14:24.805324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.805474 (+   150us) service_pool.cc:224] Handling call
0709 12:14:24.805813 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:24.806179 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.806206 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:24.806784 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 12:14:24.964897 (+158113us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:25.047566 (+ 82669us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.047905 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.048007 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.102877 (+1054870us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b1800389380. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:26.465495 (+362618us) raft_consensus.cc:1685] finished
0709 12:14:26.465596 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.465727 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68486,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1097,"raft.queue_time_us":9948,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":2970,"spinlock_wait_cycles":10855296,"thread_start_us":111865,"threads_started":1,"wal-append.queue_time_us":181996}
I20260709 12:14:26.504777 23387 rpcz_store.cc:276] 0709 12:14:24.956474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.995465 (+ 38991us) service_pool.cc:224] Handling call
0709 12:14:24.995810 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.046288 (+ 50478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.046319 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:25.046365 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:25.046395 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.504123 (+1457728us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b54000a7b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:26.504172 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.504333 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115592448}
I20260709 12:14:26.509261 26211 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:26.507158 26600 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.513805 26470 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.515308 26600 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.515815 26600 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.506626 26572 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.518399 26600 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:26.518839 26520 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.528847 26555 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.506976 24008 rpcz_store.cc:276] 0709 12:14:25.171829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.171985 (+   156us) service_pool.cc:224] Handling call
0709 12:14:25.172308 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.172650 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.172677 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.173191 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.383729 (+210538us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:25.874992 (+491263us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.927387 (+ 52395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.927482 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.460326 (+532844us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:26.506321 (+ 45995us) raft_consensus.cc:1685] finished
0709 12:14:26.506423 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.506541 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52022,"prepare.queue_time_us":1066892,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":65,"raft.queue_time_us":195048,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":23583,"spinlock_wait_cycles":28473728,"thread_start_us":677748,"threads_started":2,"wal-append.queue_time_us":392668}
I20260709 12:14:26.500128 24009 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:26.492408 24012 rpcz_store.cc:276] 0709 12:14:25.065504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.065678 (+   174us) service_pool.cc:224] Handling call
0709 12:14:25.066094 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.224898 (+158804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.462512 (+1237614us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54006def54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:26.462687 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319579392}
I20260709 12:14:26.501744 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2033) took 1280 ms. Trace:
W20260709 12:14:26.501819 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 43837.4us
W20260709 12:14:26.539012 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 69447.3us
I20260709 12:14:26.540573 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.538993 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.542151 24012 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.536295 23313 rpcz_store.cc:276] 0709 12:14:25.225630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.237940 (+ 12310us) service_pool.cc:224] Handling call
0709 12:14:25.238244 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.568342 (+330098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.568894 (+   552us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.581285 (+ 12391us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.489862 (+908577us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b180041f200. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1
0709 12:14:26.490068 (+   206us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:26.490122 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.490230 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.490272 (+    42us) raft_consensus.cc:1685] finished
0709 12:14:26.490287 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.501660 (+ 11373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17613,"raft.queue_time_us":502131,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":42915,"spinlock_wait_cycles":16697088,"thread_start_us":420428,"threads_started":2,"wal-append.queue_time_us":522593}
I20260709 12:14:26.549751 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.552188 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.553403 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.563445 24012 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:26.565104 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.544342 26525 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:26.565936 26617 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.567282 26525 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:26.557672 25950 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:26.568037 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2034) took 1342 ms. Trace:
I20260709 12:14:26.554103 26502 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.568223 26525 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.569628 26606 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.570600 26635 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:26.571008 26635 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.568454 23315 rpcz_store.cc:276] 0709 12:14:25.226050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.226302 (+   252us) service_pool.cc:224] Handling call
0709 12:14:25.284613 (+ 58311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.530147 (+245534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.530210 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.531834 (+  1624us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.567448 (+1035614us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b180041ed20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1
0709 12:14:26.567602 (+   154us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:26.567636 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.567717 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.567755 (+    38us) raft_consensus.cc:1685] finished
0709 12:14:26.567766 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.567980 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34880,"raft.queue_time_us":483355,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":32498,"spinlock_wait_cycles":30330240,"thread_start_us":532845,"threads_started":2,"wal-append.queue_time_us":438795}
I20260709 12:14:26.549230 26600 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:26.603447 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50112.9us
I20260709 12:14:26.570184 26606 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.604315 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.605897 26606 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.606061 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.607091 26600 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:14:26.607215 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.607767 26600 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:26.608655 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.572382 26635 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.609403 26502 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:26.609680 26635 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:26.602468 23219 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.615144 23219 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 5, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 5 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.618698 26634 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:26.619181 26634 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:26.617143 23219 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.574494 26672 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:26.621310 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.620136 26634 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.623369 23219 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 2 to 8, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.632975 23219 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 4 to 7, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 7 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.635149 23219 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 5 to 7, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:14:26.636631 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.641526 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:26.646819 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.647282 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.647935 23465 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:26.649086 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.650242 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.650846 23467 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:26.650991 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.626820 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.636838 23219 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.642066 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.651574 24012 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:26.652420 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.642725 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.652215 23219 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 6, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.643556 24010 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:26.644415 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.653580 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.653277 24008 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:26.653822 23219 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 4 to 6, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.655336 23219 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.656080 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.653878 23462 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:26.657779 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.658319 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.658937 23321 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:26.652313 26612 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.659642 26504 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:26.660658 26612 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.660982 26504 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.658318 23462 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:26.656949 23219 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.632834 26502 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:26.665681 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.666392 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.666106 23219 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.669457 23219 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 6, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:26.671095 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.671192 24008 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:26.670440 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.673944 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.674640 23468 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:26.674830 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.643750 26672 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.675551 23463 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.675499 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.678354 23467 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:26.680395 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.681068 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.674031 23219 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 3 to 6, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.655936 26600 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:26.674932 26612 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.680070 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.666617 26211 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:26.672339 26504 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.677749 26606 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:14:26.662639 25950 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.647342 26450 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:26.695909 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.687824 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.689366 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.720654 23387 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:26.687824 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.689090 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.704239 24008 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:14:26.683945 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.685200 23393 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:26.699713 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 515) took 1744 ms. Trace:
I20260709 12:14:26.693688 23463 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:26.712251 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.725894 24010 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:26.712487 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.726639 23465 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:26.725303 23394 rpcz_store.cc:276] 0709 12:14:24.955487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.955622 (+   135us) service_pool.cc:224] Handling call
0709 12:14:24.969047 (+ 13425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.969505 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.969546 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.991775 (+ 22229us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.379632 (+387857us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:25.427646 (+ 48014us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.673129 (+1245483us) spinlock_profiling.cc:243] Waited 8.12 ms on lock 0x7b180047e2a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:14:26.679151 (+  6022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.679222 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.679261 (+    39us) raft_consensus.cc:1685] finished
0709 12:14:26.699290 (+ 20029us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.699624 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43584,"prepare.queue_time_us":74189,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":32464,"raft.queue_time_us":1252248,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":51299,"spinlock_wait_cycles":18679552,"thread_start_us":340949,"threads_started":1,"wal-append.queue_time_us":554090}
I20260709 12:14:26.682925 26617 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.675954 26525 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.713721 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.734134 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:26.715266 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.715341 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.735546 23396 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:26.715466 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.714437 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.737008 23461 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:14:26.699155 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.667268 23462 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:26.678274 26634 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.710124 26502 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.715131 26520 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:26.694778 26548 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:26.719178 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.730336 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.655880 26662 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:26.698072 26608 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.664602 26355 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 12:14:26.731182 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.731930 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:26.719266 26605 raft_consensus.cc:993] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:26.715976 26211 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.804987 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.846192 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:26.760813 23462 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:26.794051 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.869524 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:26.869704 24016 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:26.749898 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.805449 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:26.870491 24016 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:26.837417 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:26.752007 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.788110 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.788504 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:26.827152 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.849125 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.828663 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.828922 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.822194 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.870559 23465 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:26.868764 24012 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:26.883381 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:26.875455 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48424.2us
I20260709 12:14:26.884802 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:26.873299 23463 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:26.886123 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.886740 23462 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:26.872484 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.894230 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.894964 23394 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:26.896056 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.900913 24009 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:26.774382 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 90210.6us
I20260709 12:14:26.885066 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.904567 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:26.904484 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:26.907104 23980 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:26.907315 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:26.908087 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.751482 26555 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:26.799103 25950 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:26.787405 26363 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.787213 26598 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:26.916414 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 99515.2us
I20260709 12:14:26.914803 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:26.731056 26612 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.177s
I20260709 12:14:26.787673 26535 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.919553 23392 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:14:26.940461 23392 pending_rounds.cc:85] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 7
I20260709 12:14:26.940713 23392 pending_rounds.cc:107] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:14:26.890952 26387 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.788486 26450 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.787019 26640 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.882597 26639 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.871398 26172 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.920683 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.921489 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.923035 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:26.923924 23465 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:26.787585 26596 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:26.872844 26211 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:26.909554 26670 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:26.937306 23462 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:26.904067 26552 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:26.903838 26520 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.882985 26594 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.940945 23392 raft_consensus.cc:2889] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:26.915129 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.993542 23392 pending_rounds.cc:107] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:14:26.990942 26450 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:14:26.993907 23392 raft_consensus.cc:2889] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:26.892234 26608 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.995576 26676 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:26.837965 26173 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.996481 26608 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.996332 26173 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.990700 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:26.938433 26635 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.998098 26635 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.946642 26665 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:26.999054 26665 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.999439 26665 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.000308 26635 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.915481 26555 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:26.926782 26128 raft_consensus.cc:1081] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:27.001135 26555 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:26.949069 26653 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.990060 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.984651 26548 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.001993 26653 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.002810 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.003608 23467 raft_consensus.cc:2410] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:27.004051 26653 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.904407 26662 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:26.848613 26672 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.951719 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.916337 26664 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:26.953505 26598 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.928362 26606 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.020727 25950 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.954934 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:26.955097 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2032) took 1734 ms. Trace:
I20260709 12:14:27.021351 25950 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.022794 26664 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.023005 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.023258 26664 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.023582 25950 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.021118 26606 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.026363 26606 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:26.956328 23468 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:26.902403 26486 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:26.945580 23463 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:26.748543 26617 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.965626 23396 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:26.965960 23389 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:26.968644 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:26.966724 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.036701 23465 raft_consensus.cc:2410] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:14:26.918346 23391 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:26.966540 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43587.9us
I20260709 12:14:26.964311 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:26.967185 26640 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.946573 26600 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:26.998445 26173 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:26.948061 26574 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:26.939540 26363 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.974797 26670 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:26.751076 26502 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:26.962123 26605 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:27.004884 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.015554 23470 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:27.018035 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:26.974558 26596 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.972908 26639 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:26.985199 26691 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:14:27.021575 23319 rpcz_store.cc:276] 0709 12:14:25.221110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.222424 (+  1314us) service_pool.cc:224] Handling call
0709 12:14:25.222858 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:26.510266 (+1287408us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b1800466060. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a966cbfb 00007ff0a967764b
0709 12:14:26.510429 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.510455 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:26.510944 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.581773 (+ 70829us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:26.655226 (+ 73453us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.676551 (+ 21325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.952784 (+276233us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b1800466060. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419
0709 12:14:26.953190 (+   406us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400241dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:26.953250 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.953296 (+    46us) raft_consensus.cc:1685] finished
0709 12:14:26.953307 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.955037 (+  1730us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20974,"prepare.queue_time_us":151122,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":9103,"raft.queue_time_us":396800,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":59879,"spinlock_wait_cycles":757229952,"thread_start_us":570129,"threads_started":3,"wal-append.queue_time_us":111792}
I20260709 12:14:26.976318 26501 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:27.050282 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:26.972756 26559 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:26.974617 26552 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:27.004622 26653 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:27.017601 26594 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.013301 26520 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.055572 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.021176 26598 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.082197 26605 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.071898 26670 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:27.065742 23461 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:27.070523 23388 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:27.062606 24013 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 3 to 5, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:26.965718 26387 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:26.965950 26535 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.075793 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.077215 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.078406 23462 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:27.038044 26589 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.127692 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.088030 23313 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:14:27.128213 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.053543 26600 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.093482 23395 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:27.055476 26574 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.096781 23319 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:27.098767 23464 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:14:26.976455 26172 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.037731 26486 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.117063 26605 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.109745 23463 raft_consensus.cc:2376] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6. Re-sending same reply.
I20260709 12:14:27.139256 23390 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:27.110610 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.140177 23317 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:14:27.059007 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 94891.3us
I20260709 12:14:27.112186 23394 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:14:27.071635 26173 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:27.004423 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.122768 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.143354 23389 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.101177 26552 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:27.122259 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 195870us
I20260709 12:14:27.124814 23388 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:14:27.128307 23312 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 5.
I20260709 12:14:27.130775 23462 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
W20260709 12:14:27.130674 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 226178us
I20260709 12:14:27.146261 23462 pending_rounds.cc:85] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 6
I20260709 12:14:27.146639 23462 pending_rounds.cc:107] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 12:14:27.146903 23462 raft_consensus.cc:2889] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:27.131476 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 204833us
I20260709 12:14:27.110021 26653 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:27.134600 23468 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:27.109035 23318 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:27.135427 26574 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.155259 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.160954 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.135025 26600 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:27.193158 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.193849 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:27.163344 26172 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.163987 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.126088 26662 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.168970 23389 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:27.170274 26617 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.174234 26535 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.176090 26640 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.177778 26594 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.178622 26486 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.179589 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.134779 26589 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.180092 26665 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.180847 26490 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.181248 26363 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.188158 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.189682 26639 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.194689 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:27.199272 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.239289 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:27.204061 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43610.7us
I20260709 12:14:27.242175 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:27.244956 23313 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:27.246215 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.204520 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.248427 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.205587 26596 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.206442 26583 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.207392 26387 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.209278 26605 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.213533 26504 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.280082 26504 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:27.159947 26653 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:27.217671 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55759.8us
I20260709 12:14:27.282245 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.282970 26639 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:27.283633 26670 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.284407 26639 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:27.252461 26490 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.284861 26639 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.220245 23391 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:27.284883 23465 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:27.285290 26639 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.286439 26490 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.220331 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.287444 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.288424 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.289134 23390 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:27.289981 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.223966 23396 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:27.141558 26173 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.291320 26173 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.291620 23396 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:27.224327 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.184458 26211 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.253021 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.296748 23394 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:27.256218 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.208851 26664 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.262691 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:27.263660 23388 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:27.263912 26363 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:27.259004 26583 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.306931 23390 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:27.264664 26520 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:27.265152 26525 consensus_meta.cc:287] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 12:14:27.265506 26458 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:27.265642 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.269385 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.254518 26128 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 12:14:27.310491 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47864.3us
I20260709 12:14:27.270264 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.270592 24009 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:27.324908 23318 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:27.270884 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.260039 26502 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:27.251753 26486 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:14:27.272022 26555 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.272737 23314 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:27.273202 26676 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:27.275040 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:27.275442 26608 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:27.277998 26665 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:27.283833 26552 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.377959 26487 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:27.379217 26211 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.379572 26552 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.288472 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:27.229110 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 238478us
I20260709 12:14:27.225602 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.380373 23391 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:27.293280 23466 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:27.275550 26600 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.306509 26583 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.257862 26634 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:27.364630 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51573.1us
I20260709 12:14:27.366194 23314 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:27.380646 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:27.388700 26612 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:27.388350 23314 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.389271 26612 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:27.389709 23314 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.389911 26612 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.259891 26698 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:27.404942 26698 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.406919 24010 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:27.407104 26698 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.407989 26555 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.409718 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.410302 23461 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.405256 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.309742 26387 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
W20260709 12:14:27.370229 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 95253.5us
I20260709 12:14:27.360388 26525 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:27.310335 26458 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.365623 26701 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:14:27.374186 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:27.430167 23320 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.300807 26605 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.375877 26574 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.376492 26670 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.360670 26470 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:27.380609 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:27.380637 26173 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.398592 23287 leader_election.cc:400] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
I20260709 12:14:27.400228 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.420679 24014 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:14:27.450541 23287 leader_election.cc:403] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:27.450845 23468 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:14:27.451139 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52730.8us
I20260709 12:14:27.418259 23312 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 3.
I20260709 12:14:27.368175 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.452292 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:27.452472 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 84541us
I20260709 12:14:27.426494 23467 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:27.454082 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.443496 26600 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.430401 26458 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.445253 26520 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.445673 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.526415 24015 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:27.416517 26634 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.446406 26664 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.446996 24007 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:27.447873 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:27.448633 26653 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:27.454260 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:27.552448 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 98244.8us
I20260709 12:14:27.553177 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:27.553772 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 105966us
I20260709 12:14:27.554032 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.552345 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.456562 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.458238 23320 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:27.558086 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:27.365370 26355 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:27.475322 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.485337 23314 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:14:27.496341 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:27.559473 23314 pending_rounds.cc:85] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 3
I20260709 12:14:27.526705 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.446112 24009 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:27.526439 23395 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:14:27.560753 23395 pending_rounds.cc:85] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 6
I20260709 12:14:27.560997 23395 pending_rounds.cc:107] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:14:27.560971 23980 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:27.561458 23395 raft_consensus.cc:2889] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.562668 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.557344 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.562546 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.473719 26525 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:27.525799 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:27.531459 23389 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:27.428654 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.631363 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.498494 23393 raft_consensus.cc:2368] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:27.559762 23314 pending_rounds.cc:107] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:14:27.632104 23314 raft_consensus.cc:2889] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.473294 26704 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:27.635924 24016 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:27.596967 23469 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:27.526660 26670 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:27.471334 26705 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:27.628378 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.646896 24015 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:27.628611 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.481854 26643 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:27.637337 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.661237 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 135577us
I20260709 12:14:27.663232 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:27.641202 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.612463 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.672619 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.676774 23316 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:27.676033 23388 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:27.682457 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:27.682786 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.677299 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.686677 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.687333 23463 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:27.629096 26561 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:27.526446 26502 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.148s
I20260709 12:14:27.630411 26525 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.634009 26606 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:27.642733 26450 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:27.535437 26470 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.144s
I20260709 12:14:27.667577 26670 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:27.673585 26672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.025s	sys 0.081s
W20260709 12:14:27.527909 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59217.4us
I20260709 12:14:27.717558 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:27.676995 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.708134 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.709398 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 597) took 1280 ms. Trace:
I20260709 12:14:27.713199 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.725740 23463 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.726271 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.676613 26617 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:14:27.677116 26548 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.011s	sys 0.032s
I20260709 12:14:27.681121 26504 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.724920 24014 rpcz_store.cc:276] 0709 12:14:26.428742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.428973 (+   231us) service_pool.cc:224] Handling call
0709 12:14:27.709212 (+1280239us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b540011e4d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:27.709337 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249704320}
I20260709 12:14:27.729382 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:27.706573 26635 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:14:27.754479 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.755331 23390 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.732262 23321 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:27.703822 26704 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:27.701346 26561 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.708920 26128 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:27.709512 26525 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.743429 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:27.716223 26670 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.717027 26606 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:27.716385 26434 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:14:27.756819 26676 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.803175 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:27.811785 23388 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:27.770501 26662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.023s	sys 0.023s
I20260709 12:14:27.687742 26600 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:14:27.774439 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:27.738014 26548 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.011s	sys 0.032s
I20260709 12:14:27.785307 23461 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:27.739760 26701 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:27.676487 26643 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.792666 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.769793 23463 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:27.804503 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.814582 23316 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:27.775183 26704 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:27.779726 26561 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:27.817087 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:27.778939 26128 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:27.827653 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53281.9us
I20260709 12:14:27.787124 26703 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:27.792135 26604 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:27.857007 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 294502us
I20260709 12:14:27.858530 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.852265 26572 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:27.897452 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1963) took 1098 ms. Trace:
I20260709 12:14:27.897907 23466 rpcz_store.cc:276] 0709 12:14:26.798589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.079463 (+280874us) service_pool.cc:224] Handling call
0709 12:14:27.079835 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.381243 (+301408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.897132 (+515889us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:27.897360 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":490459008}
I20260709 12:14:27.794574 26605 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:27.861351 24012 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.015s	sys 0.023s
I20260709 12:14:27.869350 23392 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:27.786844 26697 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 12:14:27.930722 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:27.890651 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73638.7us
I20260709 12:14:27.872085 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.865756 23388 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:27.869359 26714 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:14:27.894546 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.894541 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.937727 23462 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:27.852345 26574 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.923547 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.924794 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 1964) took 1126 ms. Trace:
I20260709 12:14:27.900777 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.832907 26704 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:27.816295 26700 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:14:27.867367 26173 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:27.929538 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:27.926874 26676 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:27.932401 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:27.983700 26676 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.984077 26676 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:27.984210 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51911.4us
I20260709 12:14:27.984994 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:27.986233 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:27.987090 26672 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:14:27.989562 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.990880 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.991648 23321 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:27.992511 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.995718 26589 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.924022 26634 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:27.998708 23982 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:27.999420 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.959913 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.932914 23463 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:28.000592 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.001160 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:28.001879 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.002835 23390 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
W20260709 12:14:27.928930 24012 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.015s	sys 0.023s
I20260709 12:14:27.933673 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:27.832954 26559 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:14:27.936512 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1965) took 1137 ms. Trace:
I20260709 12:14:27.944422 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:27.944444 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:28.005897 23470 rpcz_store.cc:276] 0709 12:14:26.799131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.090612 (+291481us) service_pool.cc:224] Handling call
0709 12:14:27.094335 (+  3723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.099211 (+  4876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:27.099260 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:27.109567 (+ 10307us) log.cc:844] Serialized 26 byte log entry
0709 12:14:27.534116 (+424549us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:27.534177 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.534286 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:27.936296 (+402010us) raft_consensus.cc:1685] finished
0709 12:14:27.936315 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:27.936453 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6775,"thread_start_us":404385,"threads_started":1,"wal-append.queue_time_us":737463}
I20260709 12:14:28.007280 23388 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:28.006444 23390 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:27.944862 23467 rpcz_store.cc:276] 0709 12:14:26.798864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.004175 (+205311us) service_pool.cc:224] Handling call
0709 12:14:27.899588 (+895413us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:27.924737 (+ 25149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648000896}
I20260709 12:14:27.945084 23464 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:27.926066 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2291) took 1011 ms. Trace:
I20260709 12:14:27.948881 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:27.955212 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:28.010303 23395 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:27.924378 26572 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:27.859704 26128 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:28.019378 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:27.938519 26605 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:27.968725 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:28.020478 24008 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:27.969905 26708 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:14:28.026705 26634 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:28.027282 26634 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:27.971589 26712 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:14:27.974046 26707 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:14:28.031406 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 532) took 1136 ms. Trace:
W20260709 12:14:27.976995 26639 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.000s
I20260709 12:14:27.983161 26600 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:28.011293 23467 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:14:28.037809 23467 pending_rounds.cc:85] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 3
I20260709 12:14:28.038084 23467 pending_rounds.cc:107] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:14:28.038372 23467 raft_consensus.cc:2889] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:28.008746 23393 rpcz_store.cc:276] 0709 12:14:26.914408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.914553 (+   145us) service_pool.cc:224] Handling call
0709 12:14:27.896832 (+982279us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b54002dfd54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:27.925998 (+ 29166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1308739840}
I20260709 12:14:27.938306 26604 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:28.012553 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.047402 23312 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:28.018868 26572 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:28.018116 26667 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.013854 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:28.050071 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:27.968906 26703 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:28.018422 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:28.057492 23463 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:27.968906 26662 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.975234 26643 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.031910 23387 rpcz_store.cc:276] 0709 12:14:26.895458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.895575 (+   117us) service_pool.cc:224] Handling call
0709 12:14:26.919289 (+ 23714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.202089 (+282800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:27.202145 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:27.202718 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:14:27.474729 (+272011us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:27.474787 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.898139 (+423352us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001978d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:27.898198 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.021612 (+123414us) raft_consensus.cc:1685] finished
0709 12:14:28.021640 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.031329 (+  9689us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59440,"raft.queue_time_us":66105,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":80,"spinlock_wait_cycles":512648192,"thread_start_us":271618,"threads_started":1,"wal-append.queue_time_us":550988}
I20260709 12:14:27.927968 26702 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.127s
I20260709 12:14:28.011284 26700 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:28.016573 26555 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:28.048521 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:27.868424 26486 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:14:28.050298 23462 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:28.170977 23462 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:28.014588 23466 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:28.022795 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:28.174324 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 151624us
I20260709 12:14:28.058092 23321 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:27.959025 26606 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.028836 26535 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:14:28.007584 26525 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.087740 26662 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.092103 26634 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.093206 26604 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.093778 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:28.099525 23463 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:14:28.057344 26672 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.047260 26639 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.048381 26600 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.051651 26458 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.190441 26701 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:28.121605 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:28.287581 24007 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.114105 24008 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:28.106812 26520 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.136303 26676 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.142400 26705 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
W20260709 12:14:28.046829 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 376297us
I20260709 12:14:28.168715 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:28.137468 26450 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:28.184001 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:28.298435 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 114544us
I20260709 12:14:28.178789 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.301765 23321 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:28.125047 26670 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.215389 23463 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:28.308913 23463 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:28.218076 23319 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:28.221647 23318 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:28.226692 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:28.228930 26277 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.001s	sys 0.032s
I20260709 12:14:28.240394 26583 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:28.225768 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:28.245738 26617 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.254294 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:28.370888 24009 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:28.259733 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:28.264094 26636 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.275681 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:28.374311 23465 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:28.374428 23387 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
W20260709 12:14:28.280531 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62316.4us
I20260709 12:14:28.282531 23462 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:14:28.282522 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 178654us
W20260709 12:14:28.283121 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.108325 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.284884 23461 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:28.285624 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2065) took 1018 ms. Trace:
I20260709 12:14:28.399670 23320 rpcz_store.cc:276] 0709 12:14:27.271383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.559060 (+287677us) service_pool.cc:224] Handling call
0709 12:14:27.644145 (+ 85085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.644387 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:28.031420 (+387033us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.285310 (+253890us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:28.285353 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:28.285396 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.285468 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.285572 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45143,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":4352,"raft.queue_time_us":244,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":8704,"thread_start_us":597689,"threads_started":2,"wal-append.queue_time_us":348855}
W20260709 12:14:28.216427 26612 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.826s	user 0.013s	sys 0.000s
I20260709 12:14:28.269416 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:28.307935 23312 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:14:28.336726 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:28.284104 26535 consensus_queue.cc:1243] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:14:28.338948 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:28.253059 26653 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:28.350737 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:28.353314 24012 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:28.396956 23469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2324) took 1724 ms (client timeout 1933 ms). Trace:
W20260709 12:14:28.405170 23469 rpcz_store.cc:269] 0709 12:14:26.673030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.673244 (+   214us) service_pool.cc:224] Handling call
0709 12:14:28.396454 (+1723210us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400405bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:28.396880 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646728960}
I20260709 12:14:28.410765 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:28.329912 26701 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:28.285780 26672 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.424238 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:28.426865 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47712.3us
I20260709 12:14:28.328137 26700 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:28.352524 26583 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:28.333695 26667 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 12:14:28.395756 26617 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.466987 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:28.470597 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59972.6us
I20260709 12:14:28.472268 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:28.370736 26743 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.430490 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 784) took 1173 ms. Trace:
I20260709 12:14:28.476263 24016 rpcz_store.cc:276] 0709 12:14:27.256875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.256996 (+   121us) service_pool.cc:224] Handling call
0709 12:14:28.430343 (+1173347us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54006702d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:28.430438 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750906752}
W20260709 12:14:28.441265 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:28.428112 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 129466us
I20260709 12:14:28.472729 23317 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:28.478166 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:28.433776 26700 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:28.425112 26605 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.488253 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:28.416297 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:28.352293 26277 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.002s	sys 0.032s
I20260709 12:14:28.433521 23463 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:28.495564 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.476886 24009 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:28.441567 26535 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.499181 23464 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:28.475628 26722 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:14:28.503867 24007 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:14:28.499068 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120493us
I20260709 12:14:28.412884 26128 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.423799 26612 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.513685 23463 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:28.428831 26636 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.517977 24016 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:28.468835 26738 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:28.441908 26598 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:14:28.527698 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36038 (request call id 1974) took 1045 ms. Trace:
I20260709 12:14:28.500626 26608 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:28.534446 23463 rpcz_store.cc:276] 0709 12:14:27.472531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.058835 (+586304us) service_pool.cc:224] Handling call
0709 12:14:28.517870 (+459035us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":547}
I20260709 12:14:28.506711 26486 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:28.551705 23388 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:28.517087 26552 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.555539 23314 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:28.544519 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:28.558192 23464 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:28.559700 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:28.567431 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 93992.4us
I20260709 12:14:28.488871 26743 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.566016 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:28.543047 26747 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:28.557466 26737 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:14:28.582139 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:28.516942 26535 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.554399 26486 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:28.559657 26744 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:28.566937 26608 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:28.487823 26450 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.619140 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:28.598703 26636 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:28.571785 26559 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:28.549957 26738 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.663115 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1606) took 1107 ms. Trace:
I20260709 12:14:28.550984 26701 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.589025 26458 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:28.567111 26552 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:28.594653 26739 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:14:28.597308 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 610) took 1175 ms. Trace:
I20260709 12:14:28.553860 26548 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:14:28.595783 26128 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:28.598363 26504 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.000s
W20260709 12:14:28.593999 26723 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:14:28.578778 26598 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:28.592561 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2069) took 1282 ms. Trace:
I20260709 12:14:28.595340 26676 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:28.594715 26728 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:14:28.591946 26486 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:28.597370 26743 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:28.593788 26727 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:14:28.527210 26583 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:28.590221 26639 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.581378 26173 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:14:28.581650 26700 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:28.588514 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 115246us
W20260709 12:14:28.660449 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83519.3us
I20260709 12:14:29.040084 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:28.734148 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 255670us
W20260709 12:14:28.667845 23315 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 746) took 1954 ms (client timeout 1567 ms). Trace:
W20260709 12:14:28.990051 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 370976us
I20260709 12:14:28.599987 26662 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.605528 26600 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.663544 23388 rpcz_store.cc:276] 0709 12:14:27.555503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.010268 (+454765us) service_pool.cc:224] Handling call
0709 12:14:28.010553 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.552171 (+541618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.662871 (+110700us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400344ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:28.663045 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244049280}
I20260709 12:14:29.073362 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:29.116559 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:28.992621 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.072024 23321 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:29.105523 24010 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:29.026239 26612 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.048033 23470 rpcz_store.cc:276] 0709 12:14:27.421834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.007500 (+585666us) service_pool.cc:224] Handling call
0709 12:14:28.011453 (+  3953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.011782 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.011809 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.031564 (+ 19755us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.285180 (+253616us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:28.285952 (+   772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.286011 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.597107 (+311096us) raft_consensus.cc:1685] finished
0709 12:14:28.597128 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.597250 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":233661,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":132,"thread_start_us":177621,"threads_started":1,"wal-append.queue_time_us":508188}
W20260709 12:14:29.124612 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51340.5us
I20260709 12:14:29.125880 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:28.601650 26735 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:29.132896 23463 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:28.978030 26704 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.974s	user 0.012s	sys 0.000s
W20260709 12:14:28.660740 26715 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.316s	user 0.001s	sys 0.001s
I20260709 12:14:28.617832 26744 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.048352 23313 rpcz_store.cc:276] 0709 12:14:27.309728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.723334 (+413606us) service_pool.cc:224] Handling call
0709 12:14:27.723733 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.129175 (+405442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.129209 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.139768 (+ 10559us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.255058 (+115290us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:28.255122 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.255277 (+   155us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.592365 (+337088us) raft_consensus.cc:1685] finished
0709 12:14:28.592387 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.592505 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1962,"raft.queue_time_us":91536,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":8909,"thread_start_us":469112,"threads_started":2,"wal-append.queue_time_us":322651}
I20260709 12:14:28.653787 26525 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.128279 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:28.722273 26733 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:29.072896 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.087491 26504 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:28.985937 26748 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
W20260709 12:14:29.047932 23315 rpcz_store.cc:269] 0709 12:14:26.713621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.713736 (+   115us) service_pool.cc:224] Handling call
0709 12:14:28.667622 (+1953886us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:28.667769 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218264448}
I20260709 12:14:29.147596 23464 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:28.695920 26561 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.001s
I20260709 12:14:28.665877 26559 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.665544 26363 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:29.146926 26612 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:28.657125 26670 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:28.852818 26730 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:28.664374 26738 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.134219 26583 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:29.141083 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:29.043329 26535 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.143922 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:29.174897 23461 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:29.176234 23461 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:14:29.107718 26598 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.147653 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:29.121840 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:29.157408 26600 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:29.016023 26355 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:29.128599 26672 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.146821 26744 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.157903 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:29.129202 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:29.165170 26612 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.137859 26606 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:29.123634 26640 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:14:29.142390 26704 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:28.791797 26608 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.140165 26743 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.173977 26594 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:29.180085 26730 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.182337 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 758) took 1873 ms. Trace:
I20260709 12:14:29.242573 23314 rpcz_store.cc:276] 0709 12:14:27.308458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.635424 (+326966us) service_pool.cc:224] Handling call
0709 12:14:27.635735 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.558806 (+923071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.182006 (+623200us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b5400275bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.182259 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114850560}
I20260709 12:14:29.187285 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:29.267640 23321 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
W20260709 12:14:29.187908 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 66144us
I20260709 12:14:29.162318 26559 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:29.272881 26596 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:29.273036 23314 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:29.129251 26605 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:14:29.273334 26596 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.274449 23314 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:29.274245 26596 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.275204 26596 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:29.276885 26747 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.001s
I20260709 12:14:29.152086 26502 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.162052 26363 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.203145 23464 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:14:29.208451 23470 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:29.170715 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:29.209894 23394 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.027s	sys 0.073s
I20260709 12:14:29.181370 26735 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:29.171830 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:29.220079 23463 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:14:29.163482 26561 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.166466 26525 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.221805 26598 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.222400 26608 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.197073 26600 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.206773 26355 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.202646 26535 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:29.226752 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 604) took 2297 ms. Trace:
I20260709 12:14:29.269310 26730 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.297784 26502 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.207868 26612 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.353866 26502 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.353672 26363 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.234874 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 548) took 1003 ms. Trace:
I20260709 12:14:29.354699 23393 rpcz_store.cc:276] 0709 12:14:28.231523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.326638 (+ 95115us) service_pool.cc:224] Handling call
0709 12:14:28.331439 (+  4801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.482777 (+151338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.482813 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.483319 (+   506us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.810266 (+326947us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:28.810309 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.810402 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.234632 (+424230us) raft_consensus.cc:1685] finished
0709 12:14:29.234653 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.234803 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157813,"raft.queue_time_us":151913,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":15880,"thread_start_us":326617,"threads_started":1,"wal-append.queue_time_us":675274}
W20260709 12:14:29.237856 26729 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:14:29.241408 26594 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.303685 26450 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:29.361739 26549 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:29.362314 26549 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.363358 26549 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.365101 26549 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.346621 26600 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:29.337699 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.985s (SENT) [suppressed 20 similar messages]
W20260709 12:14:29.285719 26754 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:14:29.277225 26617 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:29.408317 23462 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.000s	sys 0.028s
I20260709 12:14:29.277313 26747 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:29.305220 26487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.495s	user 0.016s	sys 0.013s
I20260709 12:14:29.278786 26704 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:29.347334 23468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.008s	sys 0.004s
W20260709 12:14:29.413654 23468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.008s	sys 0.004s
I20260709 12:14:29.351854 26520 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:29.354178 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 608) took 1010 ms. Trace:
I20260709 12:14:29.127748 26458 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:14:29.243250 26743 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.347764 24013 rpcz_store.cc:276] 0709 12:14:26.928861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.929017 (+   156us) service_pool.cc:224] Handling call
0709 12:14:26.947112 (+ 18095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.060016 (+112904us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:27.158155 (+ 98139us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.087332 (+929177us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.090058 (+  2726us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.103715 (+ 13657us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.103860 (+   145us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:28.103890 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.103957 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.226529 (+1122572us) raft_consensus.cc:1685] finished
0709 12:14:29.226551 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.226669 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14853,"prepare.queue_time_us":357175,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":32027,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":13270,"thread_start_us":902336,"threads_started":3,"wal-append.queue_time_us":935659}
I20260709 12:14:29.343438 26525 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:29.230719 26572 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.144s	user 0.012s	sys 0.000s
I20260709 12:14:29.416319 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 869) took 1951 ms. Trace:
I20260709 12:14:29.416359 24010 rpcz_store.cc:276] 0709 12:14:28.343747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.445586 (+101839us) service_pool.cc:224] Handling call
0709 12:14:28.473548 (+ 27962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.123535 (+649987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.353935 (+230400us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54006bce54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.354115 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452874624}
W20260709 12:14:29.413218 26487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.613s	user 0.016s	sys 0.013s
W20260709 12:14:29.293404 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 121693us
I20260709 12:14:29.243417 23317 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:29.294672 23320 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:29.344229 23361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.016s	sys 0.039s
I20260709 12:14:29.270690 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:29.346494 23390 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:14:29.290686 23394 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.030s	sys 0.073s
W20260709 12:14:29.408816 23462 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.000s	sys 0.028s
W20260709 12:14:29.388767 23469 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.005s	sys 0.015s
W20260709 12:14:29.410530 23466 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.010s	sys 0.019s
I20260709 12:14:29.709234 26450 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.712129 26355 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.711923 26450 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.481909 26486 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.713359 26355 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.418788 26520 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.737051 26355 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.407863 26363 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:29.676487 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 305786us
I20260709 12:14:29.742072 26486 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:29.701298 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.724432 26450 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:29.722834 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 614) took 2257 ms (client timeout 2108 ms). Trace:
I20260709 12:14:29.743364 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1989) took 1969 ms. Trace:
I20260709 12:14:29.590840 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1530) took 1644 ms. Trace:
I20260709 12:14:29.744618 23316 rpcz_store.cc:276] 0709 12:14:27.945945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.243960 (+298015us) service_pool.cc:224] Handling call
0709 12:14:28.244383 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.487439 (+243056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.487470 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:28.487508 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:28.487536 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.589873 (+1102337us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54000ecc54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.590005 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.590479 (+   474us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12908,"raft.queue_time_us":25623,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":25479,"spinlock_wait_cycles":528497024}
W20260709 12:14:29.743826 23465 rpcz_store.cc:269] 0709 12:14:27.465727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.707824 (+242097us) service_pool.cc:224] Handling call
0709 12:14:29.722213 (+2014389us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54002256d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:29.722516 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1492584704}
I20260709 12:14:29.700440 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:29.748039 26502 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.752313 26502 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.712141 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1986) took 1981 ms. Trace:
I20260709 12:14:29.430804 24014 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:29.755138 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 607) took 1411 ms. Trace:
I20260709 12:14:29.702570 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:29.755864 26502 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.756404 23319 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:29.756570 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:29.757261 23470 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:29.743892 23462 rpcz_store.cc:276] 0709 12:14:27.773588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.376205 (+602617us) service_pool.cc:224] Handling call
0709 12:14:28.376593 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.376876 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:28.518572 (+141696us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.742602 (+1224030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.742736 (+   134us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:29.742858 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.742958 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.743213 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":207147,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":308456,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9543,"thread_start_us":1307248,"threads_started":3,"wal-append.queue_time_us":699979}
I20260709 12:14:29.760221 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:29.741214 24015 rpcz_store.cc:276] 0709 12:14:27.464593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.681184 (+216591us) service_pool.cc:224] Handling call
0709 12:14:27.709127 (+ 27943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.709393 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:27.709424 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:27.709910 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.156749 (+446839us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:28.156806 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.156904 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.416101 (+1259197us) raft_consensus.cc:1685] finished
0709 12:14:29.416126 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.416264 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":446489,"threads_started":1,"wal-append.queue_time_us":1070010}
I20260709 12:14:29.753588 23466 rpcz_store.cc:276] 0709 12:14:27.730951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.184391 (+453440us) service_pool.cc:224] Handling call
0709 12:14:28.277869 (+ 93478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.278123 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:14:28.528431 (+250308us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.233970 (+705539us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:29.711120 (+477150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.711311 (+   191us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:29.711485 (+   174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.711681 (+   196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.711914 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3404,"prepare.queue_time_us":274316,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":3895,"raft.queue_time_us":244821,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":4583,"spinlock_wait_cycles":7148800,"thread_start_us":1414524,"threads_started":3,"wal-append.queue_time_us":635526}
I20260709 12:14:29.755511 24014 rpcz_store.cc:276] 0709 12:14:28.343412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.430963 (+ 87551us) service_pool.cc:224] Handling call
0709 12:14:28.468448 (+ 37485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.753939 (+1285491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.754715 (+   776us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b54006eebd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.754940 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212626176}
W20260709 12:14:29.658300 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 320509us
I20260709 12:14:29.756661 26502 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:29.741555 23469 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.005s	sys 0.018s
I20260709 12:14:29.716508 26128 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.427006 26594 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.784193 26128 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.785070 26594 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:29.762625 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2076) took 1282 ms. Trace:
I20260709 12:14:29.763007 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:29.750712 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1609) took 2194 ms. Trace:
I20260709 12:14:29.766882 23465 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:29.700937 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1987) took 1963 ms. Trace:
I20260709 12:14:29.514101 26552 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.528216 26559 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.489107 26561 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.534896 26600 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.772126 26525 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.709736 26504 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.769393 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:29.748554 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:29.643918 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.740887 26355 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.600307 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2077) took 1114 ms. Trace:
I20260709 12:14:29.577597 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:29.773918 26572 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.791306 23390 rpcz_store.cc:276] 0709 12:14:27.556390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.015018 (+458628us) service_pool.cc:224] Handling call
0709 12:14:28.016242 (+  1224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.743727 (+1727485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.750142 (+  6415us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54000a8f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.750498 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3058751872}
I20260709 12:14:29.805590 23312 rpcz_store.cc:276] 0709 12:14:28.492869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.223735 (+730866us) service_pool.cc:224] Handling call
0709 12:14:29.224008 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.580418 (+356410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.580542 (+   124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.580647 (+   105us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:29.580743 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.599472 (+ 18729us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b5400241dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.599529 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.600121 (+   592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817550080}
I20260709 12:14:29.812754 26764 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.813589 26764 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.773757 26606 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.791342 23317 rpcz_store.cc:276] 0709 12:14:28.479822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.501619 (+ 21797us) service_pool.cc:224] Handling call
0709 12:14:28.523658 (+ 22039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.761424 (+1237766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.762197 (+   773us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b5400296154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:29.762445 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1654717312}
I20260709 12:14:29.803115 26738 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.797132 23468 rpcz_store.cc:276] 0709 12:14:27.737213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.274463 (+537250us) service_pool.cc:224] Handling call
0709 12:14:28.274852 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.275111 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:28.609723 (+334612us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.699805 (+1090082us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b180031e280. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:14:29.700189 (+   384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.700303 (+   114us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:29.700422 (+   119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.700536 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.700714 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224684,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":46,"spinlock_wait_cycles":4652928,"thread_start_us":1139419,"threads_started":2,"wal-append.queue_time_us":804111}
I20260709 12:14:29.471542 26643 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:29.808708 26450 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.832680 26504 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.839375 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:29.840334 23315 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:29.840598 26600 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.842087 23389 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:29.498049 26743 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.801879 26672 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:14:29.844594 26606 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.846625 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 616) took 2372 ms. Trace:
I20260709 12:14:29.847257 26606 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.847167 23467 rpcz_store.cc:276] 0709 12:14:27.474337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.087528 (+613191us) service_pool.cc:224] Handling call
0709 12:14:28.103743 (+ 16215us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:28.283296 (+179553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.283337 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:28.286732 (+  3395us) log.cc:844] Serialized 52 byte log entry
0709 12:14:28.561204 (+274472us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:28.584249 (+ 23045us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.584432 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.584500 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.514742 (+930242us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b1800332bc0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:29.846055 (+331313us) raft_consensus.cc:1685] finished
0709 12:14:29.846222 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.846427 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29224,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":4450,"raft.queue_time_us":583748,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":39985,"spinlock_wait_cycles":24406272,"thread_start_us":286027,"threads_started":2,"wal-append.queue_time_us":652594}
W20260709 12:14:29.849949 23361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.021s	sys 0.048s
I20260709 12:14:29.850277 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:29.850605 26128 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.850800 26608 raft_consensus.cc:2764] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:14:29.850986 26572 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:29.851570 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 103304us
I20260709 12:14:29.802273 26502 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.852272 26743 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:29.852303 26653 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:29.510422 26612 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.860736 23319 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:29.867337 26559 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:29.869261 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.887087 23470 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:29.890210 23389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 542) took 2377 ms (client timeout 2136 ms). Trace:
I20260709 12:14:29.891234 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:29.895179 23389 rpcz_store.cc:269] 0709 12:14:27.513114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.933231 (+420117us) service_pool.cc:224] Handling call
0709 12:14:29.889831 (+1956600us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54003453d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:29.890045 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4207359744}
I20260709 12:14:29.882989 26743 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.881035 26504 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.902544 26764 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.917364 26670 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.909866 26743 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:29.925549 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 165545us
I20260709 12:14:29.918952 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2078) took 1322 ms. Trace:
I20260709 12:14:29.919092 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:29.920341 26502 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.920639 26643 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:29.921633 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 158934us
I20260709 12:14:29.922039 26355 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:29.922744 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.922637 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 651890us
I20260709 12:14:29.923255 26634 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:29.917797 26667 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:29.930510 26643 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.506919 26744 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.937817 26670 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.942281 26634 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.926147 23321 rpcz_store.cc:276] 0709 12:14:28.512452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.268509 (+756057us) service_pool.cc:224] Handling call
0709 12:14:29.268900 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.269251 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.269286 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.269765 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:14:29.606263 (+336498us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:29.606381 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.606509 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.804721 (+198212us) raft_consensus.cc:1685] finished
0709 12:14:29.804743 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.834604 (+ 29861us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2061,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":335793,"threads_started":1,"wal-append.queue_time_us":210300}
I20260709 12:14:29.945173 26572 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.944970 26355 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.940692 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2079) took 1315 ms. Trace:
I20260709 12:14:29.947960 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:29.948378 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.914948 26450 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.903599 26738 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.947149 23320 rpcz_store.cc:276] 0709 12:14:28.516589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.153955 (+637366us) service_pool.cc:224] Handling call
0709 12:14:29.827125 (+673170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:29.956476 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:29.940265 26608 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:29.965175 26552 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.965762 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:29.967633 26676 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:29.969692 26572 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.968319 26752 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:14:29.969043 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:29.981809 23287 leader_election.cc:400] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:29.981734 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:29.983057 23390 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:29.982050 23389 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:29.984004 23389 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:29.984313 26572 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:29.985172 26552 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:29.985384 23389 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:29.986769 23389 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:29.967648 26700 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:29.990056 26744 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:29.990573 26752 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:29.987751 26552 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:29.992046 23315 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 4.
I20260709 12:14:29.991086 26752 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:29.975337 26676 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:29.999214 26662 raft_consensus.cc:2764] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:14:29.999356 26676 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:29.963729 26667 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:30.018159 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 1995) took 1895 ms. Trace:
I20260709 12:14:30.036934 23320 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:30.037545 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:30.014228 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.009670 26608 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.039228 23390 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:30.040925 26608 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.016796 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:30.021605 26667 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.026729 26634 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:30.052560 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:30.055269 26667 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:30.019409 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.004472 26502 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:30.068270 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:30.069381 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.069736 26726 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:30.023196 26555 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.070139 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.070478 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.071379 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.071477 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.068816 26743 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.073235 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.015239 26612 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.073442 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.074450 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.077298 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.077581 26608 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:30.041754 26747 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:30.078058 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.078848 26355 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:30.079470 26363 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:30.080343 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:30.084192 26450 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.089944 26643 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:30.091030 23389 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
W20260709 12:14:30.093840 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 114762us
I20260709 12:14:30.072338 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.093824 23388 raft_consensus.cc:2376] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7. Re-sending same reply.
I20260709 12:14:30.035508 23469 rpcz_store.cc:276] 0709 12:14:28.122733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.482082 (+359349us) service_pool.cc:224] Handling call
0709 12:14:28.482403 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.782447 (+1300044us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:29.782946 (+   499us) log.cc:844] Serialized 14 byte log entry
0709 12:14:29.870978 (+ 88032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:29.871047 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:29.871113 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.017857 (+146744us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54001068d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:30.017907 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.018091 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4676,"prepare.queue_time_us":744,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":500372,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":17715,"spinlock_wait_cycles":239390592,"thread_start_us":1232146,"threads_started":2}
I20260709 12:14:30.100239 23387 raft_consensus.cc:2368] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:30.108319 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.108348 23392 raft_consensus.cc:2368] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:30.110577 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.106240 26555 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.090764 26502 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:30.093905 26726 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:30.146049 23391 raft_consensus.cc:2368] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
W20260709 12:14:30.171880 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66052us
I20260709 12:14:30.172956 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:30.173728 26793 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:14:30.179442 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.180102 23315 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:30.180811 26520 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.183174 26555 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:30.183595 26520 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:30.183822 26559 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:30.047253 26700 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.188273 26525 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.188889 26525 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.191180 23321 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:30.191535 23321 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:30.192111 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.192318 23321 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.193975 23312 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:30.195413 23316 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:30.195770 23316 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:30.196578 23316 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.197697 26782 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:30.197996 26738 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.198274 23316 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:30.198495 26600 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:14:30.199308 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:30.198303 26782 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.199419 26600 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:30.200788 26600 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:30.201877 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 121650us
I20260709 12:14:30.201793 26600 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.203349 26600 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:30.202237 26782 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:30.203382 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:30.142112 26502 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:30.205163 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:30.206015 26608 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:30.206673 26608 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:30.209694 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.193296 23321 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:30.170395 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.212316 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.216516 23470 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:30.107072 26670 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.142382 26726 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:30.214175 23436 connection.cc:256] RPC call timeout handler was delayed by -1.0485525229998984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 12:14:30.224648 23392 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:30.225891 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:30.228647 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.230646 23320 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:30.222339 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.233798 23396 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:30.142198 26747 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:30.248172 26555 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:30.309077 26555 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:30.309544 26555 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:30.309633 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73563.2us
I20260709 12:14:30.311779 26799 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:30.250545 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.257560 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.258965 26700 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:30.314507 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.315820 26747 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.317924 26676 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.320258 26676 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:30.266235 26743 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.269114 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1612) took 1827 ms. Trace:
I20260709 12:14:30.366289 23394 rpcz_store.cc:276] 0709 12:14:28.442040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.442521 (+   481us) service_pool.cc:224] Handling call
0709 12:14:28.442779 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.906446 (+1463667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.910266 (+  3820us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.946457 (+ 36191us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.167526 (+221069us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:30.167565 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.242398 (+ 74833us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540036d654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:30.242441 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.268699 (+ 26258us) raft_consensus.cc:1685] finished
0709 12:14:30.268718 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.268956 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67338,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":32992,"spinlock_wait_cycles":491301120,"thread_start_us":1282265,"threads_started":2,"wal-append.queue_time_us":256946}
I20260709 12:14:30.272002 26752 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.374611 26726 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:30.272982 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.385321 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.273576 23217 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:14:30.277689 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.289460 26594 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:30.294325 26128 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:30.295619 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:30.296145 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 71861.8us
I20260709 12:14:30.307799 26600 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:30.322053 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 100616us
I20260709 12:14:30.324833 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.325472 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.327390 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.328116 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.328418 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.329342 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.391979 23468 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.336141 23219 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 218) took 10138 ms. Trace:
I20260709 12:14:30.342986 26555 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.395038 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:30.404326 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.262699 26525 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.314777 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:30.363099 26502 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.272725 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:30.405669 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 327361us
I20260709 12:14:30.408202 23316 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 12:14:30.413362 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 117862us
I20260709 12:14:30.409006 23316 pending_rounds.cc:85] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:14:30.415263 23470 raft_consensus.cc:2376] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5. Re-sending same reply.
I20260709 12:14:30.416852 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.417721 23321 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:14:30.420394 23464 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:30.424696 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110037us
I20260709 12:14:30.418321 23321 pending_rounds.cc:85] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:14:30.435397 26583 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:30.436798 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.437824 23320 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.392663 23219 rpcz_store.cc:276] 0709 12:14:20.198001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.220527 (+ 22526us) service_pool.cc:224] Handling call
0709 12:14:30.335883 (+10115356us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:27.022022 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:27.022587 (+   565us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:27.022637 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:14:27.022665 (+    28us) tablet.cc:662] Decoding operations
0709 12:14:27.023927 (+  1262us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:27.023985 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:27.024010 (+    25us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:14:27.025098 (+  1088us) tablet.cc:701] Row locks acquired
0709 12:14:27.025118 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:14:27.025272 (+   154us) write_op.cc:270] Start()
0709 12:14:27.025395 (+   123us) write_op.cc:276] Timestamp: P: 1783599267025215 usec, L: 0
0709 12:14:27.025416 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:14:27.133891 (+108475us) log.cc:844] Serialized 5558 byte log entry
0709 12:14:28.593603 (+1459712us) op_driver.cc:464] REPLICATION: finished
0709 12:14:29.042253 (+448650us) write_op.cc:301] APPLY: starting
0709 12:14:29.042363 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:29.042747 (+   384us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:29.042911 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:29.268845 (+225934us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:29.269093 (+   248us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:14:29.269134 (+    41us) write_op.cc:312] APPLY: finished
0709 12:14:29.291123 (+ 21989us) log.cc:844] Serialized 101 byte log entry
0709 12:14:29.337573 (+ 46450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:29.348070 (+ 10497us) write_op.cc:454] Released schema lock
0709 12:14:29.358117 (+ 10047us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:29.420675 (+ 62558us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:29.427197 (+  6522us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1175335,"reported_tablets":32,"spinlock_wait_cycles":15616,"child_traces":[["op",{"apply.queue_time_us":448379,"apply.run_cpu_time_us":6057,"apply.run_wall_time_us":368307,"num_ops":14,"prepare.queue_time_us":237163,"prepare.run_cpu_time_us":37416,"prepare.run_wall_time_us":654835,"raft.queue_time_us":463684,"raft.run_cpu_time_us":8774,"raft.run_wall_time_us":259094,"replication_time_us":1568027,"thread_start_us":1238224,"threads_started":4,"wal-append.queue_time_us":641171,"wal-append.run_cpu_time_us":30311,"wal-append.run_wall_time_us":2564341}]]}
I20260709 12:14:30.427954 23321 pending_rounds.cc:107] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:14:30.442559 23321 raft_consensus.cc:2889] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:30.414755 23316 pending_rounds.cc:107] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:14:30.443413 23316 raft_consensus.cc:2889] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:30.441516 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:30.451012 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 178371us
I20260709 12:14:30.453279 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.440725 26355 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:30.455658 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.456632 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.462471 23314 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:30.460132 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:30.464267 24012 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:30.444555 26583 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:30.465741 26583 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:30.460814 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.458917 26525 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:30.482583 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:30.485272 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:30.461211 26355 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:30.488786 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:30.489334 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.446061 26799 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:14:30.470008 26552 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.441135 26600 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.474164 26700 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:30.484546 26726 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.499450 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.499866 24013 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:30.492749 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:30.479612 26743 raft_consensus.cc:2764] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:14:30.507314 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.481837 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 4, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 4 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:30.509238 23312 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:30.494355 26355 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:30.514813 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91630.7us
I20260709 12:14:30.515419 23316 raft_consensus.cc:2410] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:30.518980 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.521714 23464 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:30.513598 26700 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:30.594868 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.595518 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.596290 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.601883 26502 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.602770 26555 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:30.604063 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.605118 23464 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:30.572693 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.606719 23316 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:30.607790 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.609946 26572 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:30.610596 26572 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:30.611097 23467 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:30.610242 23217 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 3 to 5, term changed from 4 to 7, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to <none>, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:30.611579 26572 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.612921 26572 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:30.578107 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.578405 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:30.577502 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56061.7us
I20260709 12:14:30.576116 26128 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:30.623670 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 107977us
I20260709 12:14:30.581166 26600 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:30.582940 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.584986 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:30.586650 26211 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:30.587598 26676 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:30.593502 26643 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:30.594612 26700 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.609877 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.578370 26747 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:30.614188 26752 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.658252 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.534704 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.620159 24012 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:14:30.621135 23468 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
W20260709 12:14:30.660414 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 75549.5us
W20260709 12:14:30.528587 26802 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:30.522969 26552 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.626600 23312 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:30.679127 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.647848 23320 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:30.679735 23465 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:30.638422 23217 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:14:30.628302 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.680958 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.670176 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.645572 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.684551 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.686828 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:30.689208 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44062.5us
I20260709 12:14:30.678762 26667 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:30.704094 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 246916us
W20260709 12:14:30.706277 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 219548us
I20260709 12:14:30.699558 23217 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 8, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:30.707690 23464 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:30.686841 26643 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:30.708936 23317 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:30.715667 23468 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.700140 26600 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:30.721777 26782 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:30.722429 26782 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.710563 26726 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.721849 26600 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.733712 26726 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.735745 26726 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:30.739578 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.724773 26782 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:30.704620 26667 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:30.727406 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.741495 26667 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:30.741700 23316 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:30.711613 26743 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:14:30.744160 26743 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:30.728286 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.745613 23317 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:30.747730 26552 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:30.748800 23463 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:30.729169 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.729947 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.750711 23320 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:30.729969 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:30.730835 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.753865 23321 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:30.754091 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:30.755848 26594 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:14:30.756358 26594 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:30.757277 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:30.757827 26549 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:30.758054 24013 raft_consensus.cc:2368] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12.
I20260709 12:14:30.757658 23217 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:30.758366 26549 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:30.759418 26549 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.759251 23217 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 4, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 4 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:30.760370 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.760989 23318 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:30.731532 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.732270 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.763574 23314 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:30.763200 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.740777 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:30.772586 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.773012 23317 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:30.774065 23312 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:30.773077 23315 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:30.752677 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:30.781519 23468 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:30.775548 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.789386 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:30.790277 23463 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:14:30.760906 23217 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 5, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 5 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:30.787545 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.791087 23469 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:30.799821 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 104791us
I20260709 12:14:30.735775 26600 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:30.828243 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.814182 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:30.843463 23462 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:30.805461 26743 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.846681 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:30.859113 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 175610us
I20260709 12:14:30.863541 23316 raft_consensus.cc:2410] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:14:30.863626 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 111917us
I20260709 12:14:30.864356 23314 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:30.860913 23217 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 6, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:30.866420 23461 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:30.867107 23217 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:14:30.870162 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.757297 26594 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:30.872381 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.877454 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:30.883647 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.883905 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.884681 23468 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:30.847237 26520 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:30.772009 26643 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.914386 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.915154 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.915535 23467 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:14:30.839887 26726 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 12:14:30.920816 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52435.8us
I20260709 12:14:30.922732 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:30.925076 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:30.929098 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 82520.3us
I20260709 12:14:30.865043 26743 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.947335 23463 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:30.952020 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:30.907986 26211 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.928658 26600 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:30.953356 23217 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 3 to 4, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:30.960767 23217 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:14:30.961366 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.954569 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:30.966807 26636 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:30.967310 26816 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:30.967648 26636 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.977643 26211 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:30.984498 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.987257 24014 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:30.992353 26355 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:30.994572 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.995645 23466 raft_consensus.cc:2368] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8.
I20260709 12:14:30.996670 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:30.997830 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:30.998585 23468 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:30.999571 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:31.001091 23468 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:31.001331 26814 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.002089 26814 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.002609 23467 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:31.004179 23467 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:31.004648 26814 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:31.009109 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:31.013547 23466 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
W20260709 12:14:31.024446 23362 connection.cc:256] RPC call timeout handler was delayed by -1.0966028509999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 12:14:31.027568 23469 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:31.027884 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:31.022467 23217 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:31.041853 23217 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 2 to 3, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:30.954524 26747 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:31.041958 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 121609us
W20260709 12:14:31.046489 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 182064us
I20260709 12:14:31.079537 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.081300 23392 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:31.084173 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.090517 23394 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:31.092835 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 138357us
I20260709 12:14:31.095594 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.098357 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.099473 24012 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:31.107756 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.046016 26128 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:31.085119 26726 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.080538 26747 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:31.109469 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:31.115047 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.127988 26520 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:31.115604 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.112193 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.129047 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.129596 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:31.138692 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.115180 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:31.138954 24013 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:31.108923 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.139568 24012 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:31.143227 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.154280 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:31.154903 26743 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:31.155332 26211 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:31.142002 26752 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:14:31.156623 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:31.157572 24013 raft_consensus.cc:2368] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8.
I20260709 12:14:31.157783 26752 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:31.158191 26752 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:31.158505 26826 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:31.080369 26667 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.159549 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.156021 26355 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:31.180933 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.194358 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.194541 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.195032 23392 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:31.195999 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:31.197739 26726 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:31.198259 26726 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:31.198305 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.199095 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.153143 26676 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:31.199443 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:31.200384 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.291174 23396 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:31.199133 23393 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:31.271560 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.159137 26752 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:31.308331 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.232829 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:31.204186 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.257143 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.225581 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:31.310876 23318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.007s	sys 0.017s
W20260709 12:14:31.324358 23318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.007s	sys 0.017s
W20260709 12:14:31.318974 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 123006us
W20260709 12:14:31.327040 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.322383 23388 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:31.336895 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.184887 26555 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:31.339881 24016 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:31.344655 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.346820 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.350252 23393 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:14:31.347141 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.350870 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.351357 23396 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:31.199692 26670 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:31.183589 26600 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.353561 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.354602 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.356369 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.360579 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:31.363700 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 275489us
I20260709 12:14:31.370416 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.375264 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:31.377111 26782 raft_consensus.cc:2764] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:31.362776 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.199342 26726 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.385035 23391 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:31.349086 26555 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.355635 26752 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:31.318264 26832 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:14:31.416149 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.350174 26636 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:31.423887 23392 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:31.426618 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:31.426774 26643 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.427361 26643 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.394827 26826 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:31.428301 26826 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.426801 23387 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:31.430419 26826 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:31.430706 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.430996 26826 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:31.431720 26846 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.431805 26826 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:31.432014 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.432503 26825 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:31.432617 26846 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:31.434059 26829 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:14:31.391932 26211 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:31.434391 23465 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:31.434648 26211 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.435032 26211 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:31.437533 26832 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:31.410933 26555 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:31.437963 26832 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:31.439083 26832 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.440321 26832 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:31.441749 23464 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:31.442235 26825 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:31.442709 26825 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:31.433128 26846 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:31.437201 24015 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:31.442727 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.463340 26818 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.463174 26846 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.463943 26818 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:31.464664 26842 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:14:31.464673 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.465879 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:31.466094 23389 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:31.466460 23317 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:31.468040 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:31.468861 23319 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:31.463963 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.471354 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:31.472174 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.472965 23392 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:31.477826 23393 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:31.357975 26634 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:14:31.485746 23388 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:31.487964 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.480630 26700 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.479688 23317 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:14:31.507478 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.409049 26600 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.519057 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:31.480229 26643 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:31.482692 26363 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:31.527642 23313 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:14:31.536237 26833 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:14:31.390832 26520 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.534508 26820 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:31.556300 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:31.556542 23314 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:14:31.526950 26363 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:31.583103 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 211014us
I20260709 12:14:31.415620 26747 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.588605 26702 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:31.592767 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.583127 26363 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:31.598532 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.606671 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 229940us
I20260709 12:14:31.516909 26700 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.579327 26667 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:31.631256 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.635205 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 151657us
I20260709 12:14:31.602560 26450 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:31.646564 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 141465us
W20260709 12:14:31.650614 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 168086us
I20260709 12:14:31.627674 26847 raft_consensus.cc:2764] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:14:31.649484 26747 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:31.691747 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:31.697117 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 172918us
I20260709 12:14:31.622666 26820 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.700454 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:31.705116 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55677.2us
I20260709 12:14:31.705750 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:31.650817 26572 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:14:31.682277 26818 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:31.727037 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:31.651229 26450 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.656502 26667 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:14:31.734997 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:31.735662 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45109.3us
I20260709 12:14:31.715806 26847 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:31.743849 23394 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.653s	user 0.006s	sys 0.005s
I20260709 12:14:31.749370 23396 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:31.755808 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49428.3us
I20260709 12:14:31.729014 26856 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:31.749395 26844 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:14:31.768828 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:31.772971 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46047.5us
I20260709 12:14:31.767153 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:31.774750 26858 raft_consensus.cc:2764] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:31.775658 26856 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:31.776070 26852 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:14:31.776934 26743 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:31.776647 26856 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.777573 26743 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:31.777904 26849 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:14:31.778565 26743 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.781978 26700 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:31.767774 26857 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:31.782467 26211 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.784603 26211 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:31.788722 26363 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.770735 26847 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:31.751153 26814 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:14:31.791612 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.792277 23388 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:31.794883 26836 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:31.795167 26831 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:14:31.798728 26854 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 12:14:31.794924 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:31.802312 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.817907 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:31.818647 23470 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:31.755077 26555 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.776558 26600 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:31.821643 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:31.822764 23315 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:31.826279 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:31.829373 23461 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:31.831203 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.831997 26836 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:31.832443 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.806988 26814 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.812709 26549 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.839814 24016 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:31.817865 26857 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:31.839273 26836 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:31.856511 26872 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:31.859967 26859 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 12:14:31.861832 26847 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:31.873351 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.874830 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:31.875408 23388 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:31.844254 26814 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:31.881239 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 112473us
I20260709 12:14:31.879724 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 803) took 1072 ms. Trace:
I20260709 12:14:31.886554 23316 rpcz_store.cc:276] 0709 12:14:30.807520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.867872 (+ 60352us) service_pool.cc:224] Handling call
0709 12:14:30.868213 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:31.216546 (+348333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.216596 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:31.309737 (+ 93141us) log.cc:844] Serialized 26 byte log entry
0709 12:14:31.491170 (+181433us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:31.491237 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.491328 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:31.798021 (+306693us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:31.869547 (+ 71526us) raft_consensus.cc:1685] finished
0709 12:14:31.869570 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.879652 (+ 10082us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":289762,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":66,"spinlock_wait_cycles":5860096,"thread_start_us":500041,"threads_started":2,"wal-append.queue_time_us":337006}
I20260709 12:14:31.880612 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:31.848044 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 130591us
I20260709 12:14:31.892967 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:31.922890 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:31.923730 23461 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:31.925521 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.925726 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:31.926198 23396 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:31.934453 23393 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:14:31.929651 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:31.939652 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:31.940514 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:31.941751 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.943567 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.945729 23388 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:31.949040 23390 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:31.949477 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:31.949604 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:31.950099 23389 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:31.930015 26676 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:31.962731 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:31.938263 26871 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:31.870271 26549 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:31.978873 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46590.5us
I20260709 12:14:31.981735 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.954537 26863 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:31.989985 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:31.961956 26744 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:31.995374 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:31.992244 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:32.004472 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:32.002605 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58695.4us
I20260709 12:14:32.007735 26820 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:32.009442 23362 leader_election.cc:400] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8.
I20260709 12:14:32.009789 23362 leader_election.cc:403] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:14:31.980854 26826 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.946875 26825 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:31.857388 26872 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.010273 26363 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:31.990622 26676 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:31.994939 26863 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.000864 26744 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:32.027400 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.001003 26871 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:31.996421 26726 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.017781 26847 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:32.046732 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63506.6us
I20260709 12:14:32.025315 26863 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:32.055245 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45940.1us
I20260709 12:14:32.059751 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:32.028908 26869 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.045608 26879 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:32.043386 26743 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:32.068243 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.043695 26744 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:32.093542 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66242.3us
I20260709 12:14:32.075651 26555 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.077689 26879 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.054874 26363 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:14:32.098737 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.100185 26450 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.100569 26572 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:32.096138 26670 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:32.102696 26865 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8.
I20260709 12:14:32.102938 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 804) took 1290 ms. Trace:
I20260709 12:14:32.113798 23320 rpcz_store.cc:276] 0709 12:14:30.812953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.889049 (+ 76096us) service_pool.cc:224] Handling call
0709 12:14:30.918700 (+ 29651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:31.247254 (+328554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.247298 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:31.247841 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:14:31.443446 (+195605us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b48000d08d8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:14:31.579567 (+136121us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:31.579610 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.579691 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:32.102551 (+522860us) raft_consensus.cc:1685] finished
0709 12:14:32.102584 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:32.102857 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":433051,"raft.run_cpu_time_us":978,"raft.run_wall_time_us":84406,"spinlock_wait_cycles":7573376,"thread_start_us":542823,"threads_started":2,"wal-append.queue_time_us":679028}
I20260709 12:14:32.103444 26846 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:32.132519 26886 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:32.023304 26826 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.133082 26886 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.133529 26886 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:32.134989 26826 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.048704 26726 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.139372 26825 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.141784 26872 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:32.149659 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.112212 26832 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.004s
W20260709 12:14:32.162154 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 111733us
I20260709 12:14:32.167138 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:32.167652 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 99525.5us
I20260709 12:14:32.105545 26879 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:32.170028 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:32.170339 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.155155 26871 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.173609 26846 raft_consensus.cc:2764] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:14:32.182657 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.186537 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:32.187078 23465 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.193945 23287 leader_election.cc:400] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:32.194788 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:32.198069 26873 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 12:14:32.199422 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:32.200245 24013 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:14:32.096791 26676 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.178095 26825 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:32.213371 23314 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:32.218322 26363 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:32.219689 26363 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:32.220151 23362 leader_election.cc:400] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
I20260709 12:14:32.220222 26363 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:32.220542 23362 leader_election.cc:403] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:14:32.184134 26832 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:32.221762 26832 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.225952 26572 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
I20260709 12:14:32.177889 26670 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:32.238723 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44003.3us
I20260709 12:14:32.242276 23465 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:32.244689 23287 leader_election.cc:400] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:32.246770 23287 leader_election.cc:403] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:14:32.245925 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54073.3us
I20260709 12:14:32.236284 26826 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.221202 26363 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.254328 23465 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:32.254570 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:32.256045 24016 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:32.263917 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.267699 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:32.268368 24014 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:32.270304 23463 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:14:32.270720 23463 pending_rounds.cc:85] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 2
I20260709 12:14:32.271261 23463 pending_rounds.cc:107] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:14:32.271598 23463 raft_consensus.cc:2889] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:32.273612 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:32.274304 24012 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:32.274619 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.275473 23467 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:32.279994 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.280886 23463 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:32.286281 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:32.249714 26846 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:32.298972 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:32.308419 23465 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:14:32.308652 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68928.1us
I20260709 12:14:32.331972 23461 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:14:32.333287 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:32.335460 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 93573.8us
I20260709 12:14:32.248836 26863 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.326486 26826 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:32.339432 26355 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:32.340617 26363 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:32.325932 26889 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.147s
W20260709 12:14:32.349861 26747 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.003s
I20260709 12:14:32.363621 24015 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:32.369635 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.352418 26846 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.370926 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.380636 23465 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:32.362083 26826 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.382594 26600 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.383251 26826 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:32.371196 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.370422 23463 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:32.390820 23318 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:32.331341 26670 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:32.391528 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 63966.5us
W20260709 12:14:32.467208 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 79229.8us
I20260709 12:14:32.468340 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.377027 26355 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:32.442597 26891 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 12:14:32.462828 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 153247us
I20260709 12:14:32.478410 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.488906 23287 leader_election.cc:400] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
W20260709 12:14:32.479779 26884 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:14:32.492149 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 191471us
I20260709 12:14:32.494141 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:32.476648 26743 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:32.482136 26747 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:32.498373 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.500208 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.506247 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.505653 23362 leader_election.cc:400] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12.
I20260709 12:14:32.507436 23362 leader_election.cc:403] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:32.500007 26600 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:32.516909 26897 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.517081 26901 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.517560 26901 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.520516 26901 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.521170 26874 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.517892 26899 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.521870 26874 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.522362 26899 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.524391 26899 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:32.524619 26874 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:32.525089 26897 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.537834 26901 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12.
I20260709 12:14:32.540809 26888 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:32.541872 26897 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.544108 26743 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.507951 26893 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.553298 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 458077us
I20260709 12:14:32.554051 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:32.541249 26888 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:32.558634 26847 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:32.515290 26355 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.565073 26908 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:32.564453 26450 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:32.568342 26450 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.568733 26450 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.569756 26549 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:32.541877 26887 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.563345 26826 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:14:32.533838 26889 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.562918 26818 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:32.564445 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:32.564558 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60068.3us
I20260709 12:14:32.578508 23362 leader_election.cc:400] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8.
I20260709 12:14:32.572256 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:32.595527 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.579596 23317 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:32.599467 23312 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:14:32.599753 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:32.566030 26862 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.008s	sys 0.002s
W20260709 12:14:32.600668 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 165417us
I20260709 12:14:32.557161 26863 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:32.565471 26888 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:32.607694 23316 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:32.577562 26889 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:32.611842 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 108131us
I20260709 12:14:32.635306 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.604835 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:32.556838 26846 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:14:32.611454 26862 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.009s	sys 0.002s
W20260709 12:14:32.574621 26856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.005s	sys 0.013s
I20260709 12:14:32.593220 26818 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:32.638510 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:32.642858 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.652092 23317 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:14:32.652835 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.653795 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.652835 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.659112 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:32.636893 26889 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.660651 23316 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
W20260709 12:14:32.650228 26856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.005s	sys 0.013s
I20260709 12:14:32.643857 26908 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.639511 26549 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.669093 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.707386 23318 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:32.708755 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.709453 23316 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:32.709136 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:32.556988 26893 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.649880 26747 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:14:32.662202 26643 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:32.720654 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.674899 26908 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.721295 23316 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:32.722589 23388 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:32.722530 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.739610 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:32.741118 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.739631 23393 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11. Re-sending same reply.
W20260709 12:14:32.746315 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 111101us
I20260709 12:14:32.743877 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.747102 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.747807 23394 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:32.749919 26814 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.008s
I20260709 12:14:32.750379 26814 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.752534 26814 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:32.754036 26907 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:14:32.719812 26909 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.760504 26909 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.760900 26909 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.711306 26913 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:32.718331 26743 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:32.764086 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:32.772898 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.737950 26908 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:32.767522 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:32.781523 23396 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:32.769207 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.769534 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.708310 26594 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:32.764040 23389 raft_consensus.cc:2368] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9.
I20260709 12:14:32.717823 26872 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:14:32.714397 26549 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.796491 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.797128 26594 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:32.797401 23393 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:32.777842 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.798347 23318 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:32.773873 26814 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.779469 24016 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:32.799149 26814 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.800120 26872 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:32.800529 26872 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:32.778707 26450 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.797937 26549 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:32.776602 26886 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.643s	user 0.014s	sys 0.008s
I20260709 12:14:32.768174 26913 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.765161 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.797567 26594 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:32.801323 26872 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.810590 26922 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.811250 26520 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.811686 26552 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.798398 26902 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.808853 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 315387us
I20260709 12:14:32.737946 26818 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.878139 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:32.878464 23321 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:32.777115 26743 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.879666 23316 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:32.879850 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:32.879663 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.880612 23388 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:32.842612 26913 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:32.880813 23387 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12. Re-sending same reply.
W20260709 12:14:32.879768 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102007us
I20260709 12:14:32.891173 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.882344 23394 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:32.891659 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:32.885363 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.887880 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.892513 23318 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:14:32.894840 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.896487 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.865242 26520 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.907131 23387 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:32.907401 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.909981 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.913508 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.914254 23315 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:32.843346 26886 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:32.894130 26872 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:32.894519 26921 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:32.919690 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:32.894762 26818 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:32.921029 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.921473 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.927255 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:32.935086 23319 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:32.918928 23387 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:32.943624 23388 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:32.944818 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.946341 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:32.910996 26869 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:32.918545 26863 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.949548 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.950264 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.925585 26871 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.952452 23469 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:14:32.954481 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.944427 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:32.971175 23387 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:32.972873 23387 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:32.971169 23464 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:32.971124 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 205993us
I20260709 12:14:32.974902 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:32.974759 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:32.976248 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:32.975857 23395 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11. Re-sending same reply.
I20260709 12:14:32.980813 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:32.951668 26846 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:32.950656 26837 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:32.866338 26922 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:32.947149 26450 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:33.010085 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 63810.7us
I20260709 12:14:32.954577 26863 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:32.878927 26902 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.076912 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:33.077643 23319 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:33.065917 26835 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.268s	user 0.006s	sys 0.013s
I20260709 12:14:32.953974 26864 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:14:32.980088 26856 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:32.925894 26594 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:33.109175 26835 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.312s	user 0.006s	sys 0.013s
W20260709 12:14:33.083467 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 130999us
I20260709 12:14:32.988109 26814 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:33.071923 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66370.1us
I20260709 12:14:33.005148 26874 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:33.002868 26747 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:33.063686 26846 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.011180 26837 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.981001 26927 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:33.081243 26670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.006s	sys 0.001s
I20260709 12:14:33.075217 26863 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:33.072737 26744 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.977s	user 0.016s	sys 0.006s
I20260709 12:14:33.085994 26450 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.957836 26869 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.124871 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:32.998165 26897 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.116s
I20260709 12:14:33.134137 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.143590 26898 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 12:14:33.144634 26930 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:33.137552 26747 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:33.180043 26747 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.109674 26894 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
W20260709 12:14:33.129021 26670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.006s	sys 0.001s
I20260709 12:14:33.153677 26744 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.137288 26927 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.129455 26837 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.167553 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.177706 26925 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.134908 26594 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:33.169603 26888 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.005s
I20260709 12:14:33.163208 26520 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:33.167382 26899 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:33.243211 26899 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.243661 26899 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.129896 26846 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.169992 26930 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.181779 26893 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.241572 23467 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:14:33.242515 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.249984 23319 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:33.209812 26743 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.248410 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.162560 26450 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.259490 24014 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:33.248562 26901 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:14:33.255302 26930 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:33.277403 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.278131 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.278784 26866 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:33.279318 26918 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.246s	user 0.000s	sys 0.003s
W20260709 12:14:33.282305 26931 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:14:33.279974 26939 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:33.283233 24014 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:14:33.284518 26450 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.240346 26925 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.290185 26832 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:33.290856 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 154580us
I20260709 12:14:33.261117 26886 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.248870 26888 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.303275 24008 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:33.262567 26743 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:14:33.299865 26879 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.113s	user 0.014s	sys 0.000s
I20260709 12:14:33.312928 26943 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:33.309875 26939 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.309998 26866 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:33.366657 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 390533us
W20260709 12:14:33.365223 26909 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.000s
W20260709 12:14:33.367270 26363 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.000s	user 0.009s	sys 0.005s
W20260709 12:14:33.380182 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68616.9us
W20260709 12:14:33.382707 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 117725us
I20260709 12:14:33.383514 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.371376 23391 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:14:33.366542 26889 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.003s
I20260709 12:14:33.401608 26922 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.402102 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.402275 26832 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:33.402827 24009 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:33.403853 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.404166 26450 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.404736 26450 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.404997 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.406751 26450 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:33.406921 26908 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.000s
I20260709 12:14:33.381481 26902 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:33.381723 24008 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:33.382436 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.408567 24009 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:33.408953 24014 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:33.383958 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.411345 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.386229 26925 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.368330 26939 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.387728 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.388180 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.389566 26355 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.391062 26874 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:33.392836 26863 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.416734 24011 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:14:33.434370 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.434638 24011 raft_consensus.cc:606] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.408393 26450 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 12:14:33.428628 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45155.4us
I20260709 12:14:33.407374 26908 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.432225 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.431849 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:33.431849 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:33.427032 26872 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.445397 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.367980 26879 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.397866 26889 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.447815 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.430516 26869 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:33.434237 26826 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.001s	sys 0.022s
I20260709 12:14:33.452322 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.456276 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.432895 26902 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:33.439666 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.459255 26924 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:14:33.465732 26890 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:33.465680 26837 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.468691 26893 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:33.502375 26893 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:33.502864 26893 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:33.504525 26818 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.558s	user 0.008s	sys 0.002s
I20260709 12:14:33.469625 26927 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.470906 26726 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.446439 26744 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.434093 26355 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.474220 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.448135 26922 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.476480 26889 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:33.454396 26913 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.004s
I20260709 12:14:33.480402 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.472718 26899 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.453264 26874 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:33.487299 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.488870 23312 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:33.417726 26909 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.524648 24012 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:33.510164 24009 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:14:33.525090 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 93306.4us
I20260709 12:14:33.526113 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.527554 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:33.516425 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 76828.7us
I20260709 12:14:33.464919 26930 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.524293 24008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:14:33.530783 24008 raft_consensus.cc:606] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:33.518850 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.532023 23394 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.532740 23394 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:33.524279 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:33.479722 26826 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.001s	sys 0.022s
I20260709 12:14:33.539153 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.482296 26950 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:33.547590 26949 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:33.550587 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70246.9us
I20260709 12:14:33.551438 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.504873 26939 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.495324 26846 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.559374 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.556129 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.556823 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.515594 26700 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:33.564332 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.545501 26744 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:33.545166 26901 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.570274 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.548833 26826 consensus_queue.cc:1243] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:14:33.545363 26952 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:33.578143 23392 raft_consensus.cc:2376] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9. Re-sending same reply.
I20260709 12:14:33.548246 26818 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:33.548163 26866 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.595885 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.596498 23394 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12. Re-sending same reply.
I20260709 12:14:33.566547 26922 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:33.595244 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.574250 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.563030 26874 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:33.570812 26636 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:33.611392 26874 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.612939 26520 raft_consensus.cc:993] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:33.613415 26520 raft_consensus.cc:1081] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:33.615696 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:14:33.616326 23316 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:33.616658 23316 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:33.618103 26863 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:33.620368 26837 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.622082 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2024) took 1345 ms. Trace:
I20260709 12:14:33.622468 23467 rpcz_store.cc:276] 0709 12:14:32.277047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.279237 (+  2190us) service_pool.cc:224] Handling call
0709 12:14:32.279648 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:33.257530 (+977882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.621819 (+364289us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b54001068d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:33.622021 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205844224}
I20260709 12:14:33.623413 26930 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.625380 26908 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:33.625505 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.626808 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:33.626906 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.627767 23392 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:33.628545 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.602912 24008 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:33.607689 26954 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:14:33.609620 26355 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:33.609702 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.521667 26913 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.611301 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.648555 26747 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:33.566164 26890 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:14:33.661279 23463 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:14:33.661731 23463 pending_rounds.cc:85] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 4
I20260709 12:14:33.661952 23463 pending_rounds.cc:107] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:14:33.662174 23463 raft_consensus.cc:2889] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:33.663326 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:33.593793 26952 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.597036 26950 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:33.684144 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:33.684917 23467 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:33.693037 24008 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:33.694195 23390 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:33.678649 26952 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:33.703850 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:33.704769 23468 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:33.649078 23316 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.562256 26949 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.716550 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.717216 23392 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:14:33.718266 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 144140us
I20260709 12:14:33.654887 24007 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:33.714416 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.719511 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.715371 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:33.730794 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:33.731858 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.737268 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.738862 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.739676 23316 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.743899 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.634645 26954 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:33.687928 26363 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:14:33.687691 26818 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:14:33.717878 26726 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.728179 26890 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:14:33.736795 26894 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:33.734840 26858 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:33.751024 26744 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:33.769918 26939 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:33.757866 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:33.747117 26925 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:33.767985 26949 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.777159 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:33.729017 26355 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:33.777980 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:33.779793 26862 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:33.780799 26951 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:33.781482 26847 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:33.786599 23462 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:33.783452 26600 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:33.792004 26956 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:14:33.791926 26864 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:33.763021 26832 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:33.795128 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:33.765130 26930 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:14:33.775490 26890 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:33.800020 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:33.788211 26925 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.785087 26744 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:33.823787 23462 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:14:33.822114 26905 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.256s	user 0.005s	sys 0.000s
I20260709 12:14:33.824925 26909 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.793004 26927 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.797120 26743 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:14:33.789139 26862 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:33.792855 26355 consensus_queue.cc:1243] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:33.768839 26954 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.776384 26726 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.851481 26954 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:33.852617 26726 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.838795 26744 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:33.804989 26670 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 12:14:33.865166 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 70154.1us
I20260709 12:14:33.845615 23392 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:33.793371 26951 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:33.836354 26925 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.859730 24008 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:33.864051 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:33.869262 23463 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:33.839865 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:33.840041 26747 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:33.860512 26927 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.872071 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 572) took 1157 ms. Trace:
I20260709 12:14:33.872454 23389 rpcz_store.cc:276] 0709 12:14:32.715013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.715170 (+   157us) service_pool.cc:224] Handling call
0709 12:14:33.871875 (+1156705us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:33.872007 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41343872}
I20260709 12:14:33.873715 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.876520 23468 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:33.877532 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.877776 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 571) took 1163 ms. Trace:
I20260709 12:14:33.878165 23390 rpcz_store.cc:276] 0709 12:14:32.714692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.714817 (+   125us) service_pool.cc:224] Handling call
0709 12:14:33.877565 (+1162748us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400345dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:33.877701 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2177687424}
I20260709 12:14:33.879222 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.879369 23388 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:14:33.880905 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.882830 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.883464 23390 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:33.884285 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.884893 23390 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:33.886230 26670 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:33.798453 26847 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:33.889343 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.890069 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 579) took 1052 ms. Trace:
I20260709 12:14:33.890380 23396 rpcz_store.cc:276] 0709 12:14:32.838060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.889086 (+1051026us) service_pool.cc:224] Handling call
0709 12:14:33.890014 (+   928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:33.891280 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.846518 26950 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:33.897912 26847 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.899132 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 582) took 1023 ms. Trace:
I20260709 12:14:33.899281 26847 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.899509 23391 rpcz_store.cc:276] 0709 12:14:32.875990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.876141 (+   151us) service_pool.cc:224] Handling call
0709 12:14:33.898899 (+1022758us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b5400415854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:33.899080 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132068224}
I20260709 12:14:33.818233 26600 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.921000 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.922400 23389 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:33.923290 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.924417 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.924943 23389 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:33.928689 23317 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:33.929100 23317 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:33.929812 23317 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.930903 23317 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:33.937927 26951 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.939277 26951 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:33.906710 23391 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:14:33.909171 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.940753 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2330) took 1147 ms. Trace:
I20260709 12:14:33.941088 23393 rpcz_store.cc:276] 0709 12:14:32.793050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.908899 (+1115849us) service_pool.cc:224] Handling call
0709 12:14:33.940694 (+ 31795us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:33.942123 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.942276 26450 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:33.943240 26450 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.943806 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:33.944495 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:33.944783 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:33.945354 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:33.946368 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:33.947053 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.946949 26924 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.947568 26924 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.948136 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:33.948793 26826 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.913705 26913 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.949725 26913 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:33.915126 26862 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.916527 26893 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:33.917865 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2360) took 1553 ms. Trace:
I20260709 12:14:33.951859 23470 rpcz_store.cc:276] 0709 12:14:32.364936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.365083 (+   147us) service_pool.cc:224] Handling call
0709 12:14:32.365457 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.365767 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.365805 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.371436 (+  5631us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.493332 (+121896us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:32.493367 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.493446 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.755504 (+1262058us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:33.917564 (+162060us) raft_consensus.cc:1685] finished
0709 12:14:33.917586 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.917804 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13582208,"thread_start_us":111145,"threads_started":1,"wal-append.queue_time_us":166661}
I20260709 12:14:33.942862 23393 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:33.954126 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:33.954710 24015 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:33.954993 24015 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:33.955610 24010 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:33.955706 24015 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:33.956890 24015 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:33.957811 23393 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:14:33.958952 26927 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:33.901577 23462 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
W20260709 12:14:33.938660 26926 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.935s	user 0.003s	sys 0.000s
I20260709 12:14:33.950057 26747 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.908536 26954 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:33.902998 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:33.971391 23319 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:33.975039 23391 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:33.979202 23390 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.982373 23391 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:33.984639 23391 pending_rounds.cc:85] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 8
I20260709 12:14:33.984910 23391 pending_rounds.cc:107] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:14:33.984975 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:33.985252 23391 raft_consensus.cc:2889] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:33.985618 23318 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:33.986554 24007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:14:33.987107 24007 raft_consensus.cc:606] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:33.991773 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:33.992475 23462 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:33.993228 26973 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.991763 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.988725 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.994827 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:33.998447 23312 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.998346 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:33.999519 23396 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:34.042310 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.041955 23317 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:14:34.059125 23317 pending_rounds.cc:85] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 4
I20260709 12:14:34.059798 23317 pending_rounds.cc:107] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:14:34.010102 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.060945 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.061672 23317 raft_consensus.cc:2889] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:34.000821 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:34.024933 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.040988 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.063232 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.014971 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.064272 23461 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:33.949360 26826 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.066583 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:33.974184 26846 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:34.069324 23391 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:33.978667 26747 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:34.071749 23389 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:34.072741 23388 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:14:33.987596 26954 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.063117 23436 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.082937 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2123) took 1210 ms. Trace:
I20260709 12:14:34.083242 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.083470 23317 rpcz_store.cc:276] 0709 12:14:32.872844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.873089 (+   245us) service_pool.cc:224] Handling call
0709 12:14:32.873533 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.928572 (+1055039us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b54000b4da0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:14:34.082631 (+154059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.082868 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1636116864}
I20260709 12:14:34.086606 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.087390 23316 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:34.085170 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.089581 23317 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:33.997422 26922 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:34.090719 23391 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:34.040707 26858 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.065883 26869 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.128756 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.135978 23389 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:34.123811 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.123421 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.126376 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.146333 23391 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:34.132917 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.104647 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.081489 26600 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.085974 26743 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:34.158783 23316 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:34.080578 26954 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.165102 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.169332 23314 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:33.997597 26952 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.100943 26670 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.127462 26913 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.183511 23312 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:34.185056 23389 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:34.190869 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.125068 26890 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:14:34.087502 26846 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.193610 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:34.160768 26572 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.125294 26908 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:14:34.201658 23317 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:34.151425 26869 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.206350 26869 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.206943 26572 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.209035 26572 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:34.209563 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.209789 26908 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:34.210196 26908 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:34.210916 26908 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.211787 23314 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:34.211686 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.213927 23321 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:14:34.215052 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.190500 26925 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.219048 26925 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.219766 23391 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:34.219901 26676 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:34.211863 26908 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:34.131923 26858 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:14:34.063575 26973 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.162771 26743 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:34.271078 23318 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:34.271862 23387 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:34.221725 26924 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:34.277251 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 186458us
I20260709 12:14:34.276234 26702 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:34.274680 26954 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:34.321559 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.203984 26913 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.330526 24015 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
W20260709 12:14:34.326261 26970 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:14:34.279882 26743 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:34.347869 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2030) took 1403 ms. Trace:
I20260709 12:14:34.348330 23466 rpcz_store.cc:276] 0709 12:14:32.944267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.944824 (+   557us) service_pool.cc:224] Handling call
0709 12:14:32.962399 (+ 17575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.605647 (+643248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.605681 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:33.605742 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:33.605785 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.347622 (+741837us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540038e5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:34.347668 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.347803 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478862592}
I20260709 12:14:34.354135 26973 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.355333 26888 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:34.359764 26913 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:34.362648 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.362905 26846 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.364674 26887 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.364838 26879 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.365293 26887 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.367337 26887 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.367427 26893 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:34.368268 26744 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:34.363090 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.369073 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:34.377003 23391 raft_consensus.cc:606] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:34.379083 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 160045us
I20260709 12:14:34.383167 23462 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:34.384776 23313 raft_consensus.cc:1217] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[4.6-12.10]   Dedup: 6.7->[8.8-12.10]
I20260709 12:14:34.319456 26702 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:34.385609 23313 pending_rounds.cc:85] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:14:34.388226 23313 pending_rounds.cc:107] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:14:34.388509 23313 raft_consensus.cc:2889] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:34.393169 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.387784 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 372888us
I20260709 12:14:34.398259 23391 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:14:34.486215 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.538439 23393 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.023s	sys 0.071s
W20260709 12:14:34.538933 23393 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.024s	sys 0.071s
W20260709 12:14:34.524217 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 111112us
I20260709 12:14:34.540275 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.544611 23462 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:34.542331 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.547410 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.393039 26954 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:34.562892 26820 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.563467 24013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:14:34.563464 26820 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.564213 24013 raft_consensus.cc:606] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.564567 24013 raft_consensus.cc:614] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:34.566154 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:34.567966 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:34.569054 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:34.556087 26973 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:14:34.588562 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:34.581728 23390 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.002s
I20260709 12:14:34.595256 23390 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:34.596472 23389 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:14:34.596930 26820 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.585520 26890 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:34.597680 26889 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.433478 26826 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.598008 26890 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.598635 26952 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.384958 26977 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.181s
I20260709 12:14:34.600073 26890 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.546132 26818 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.603719 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 599) took 1040 ms. Trace:
I20260709 12:14:34.605767 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.604163 23388 rpcz_store.cc:276] 0709 12:14:33.563531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.979464 (+415933us) service_pool.cc:224] Handling call
0709 12:14:34.072408 (+ 92944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.073208 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.603518 (+530310us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b540036cc50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:34.603649 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189399424}
I20260709 12:14:34.550791 26924 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:34.617264 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.622759 23466 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:34.582949 26976 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:34.571048 26846 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:34.626644 23466 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:34.630445 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 525861us
I20260709 12:14:34.628966 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.615785 26924 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:34.639173 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.569667 26954 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.642379 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.644058 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:34.626520 26976 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.627838 26846 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:34.652051 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:34.636476 26924 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:34.652817 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 105469us
I20260709 12:14:34.653126 23466 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:34.647934 26949 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:34.651228 26976 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:34.648947 26846 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:34.678843 23461 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:34.682283 26954 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.687350 26897 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:34.687799 26897 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:34.688625 26897 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.690555 26702 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.691637 26702 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:34.692694 26951 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.004s
I20260709 12:14:34.693457 23320 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:34.676896 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.695231 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.696946 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.697904 23461 raft_consensus.cc:2368] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9.
I20260709 12:14:34.647532 26952 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:14:34.661455 26865 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:34.676124 26925 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:34.702848 26747 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.000s
W20260709 12:14:34.703147 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.704218 26913 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:34.707156 26865 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:34.707656 26954 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:34.708379 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.718861 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.720237 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.723984 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.724633 23390 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:34.725078 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.725638 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.726336 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.726805 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.728113 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.730063 23470 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:34.710922 26897 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:34.730341 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.729640 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.711526 26951 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:34.731139 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:34.731561 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.736866 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.738319 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.717681 26826 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:34.721001 26925 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:34.750166 26885 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.751991 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.751866 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.752933 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.763940 23394 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:34.758291 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.710767 26747 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.744854 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.751024 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:34.738292 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.764606 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.764842 24012 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:34.732534 26913 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:34.771086 23982 leader_election.cc:400] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9.
I20260709 12:14:34.767632 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.771838 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.729602 26973 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:34.743320 26987 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.782755 23462 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:34.741648 26950 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.713647 26980 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:14:34.780485 23982 leader_election.cc:403] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:14:34.741508 26909 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.758319 26864 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:34.797410 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.781978 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.782516 26893 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.769577 26875 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:14:34.777169 26832 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:14:34.786216 26747 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.807879 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.786666 26987 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.787420 26973 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.814097 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.779322 26983 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:14:34.775496 26888 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:34.775671 26837 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:14:34.790238 23465 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:34.762169 26908 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.747131 26988 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:34.766950 26967 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.808890 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.812058 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:34.805693 26450 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.794979 26211 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:34.822954 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:34.821800 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95540.9us
I20260709 12:14:34.827431 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:34.827823 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.813072 26676 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:34.807157 26954 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.814721 24016 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:34.813174 26950 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:34.773872 26885 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.814349 26887 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:34.828685 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:34.833571 26990 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:14:34.839720 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.823362 26951 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:34.797098 26925 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:34.847630 26981 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:14:34.841817 26950 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:34.845865 26450 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:34.850474 26973 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.830652 26893 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.829613 26747 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:34.855228 23313 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:14:34.862883 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.856451 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.858098 26987 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:34.859441 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:34.863839 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.844200 26988 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:34.812426 26865 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.861008 23319 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:34.844555 26887 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:34.867823 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.864634 23462 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:34.873708 26747 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:34.874614 26747 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:34.867697 26450 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:34.831323 26913 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:34.876564 26450 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.879200 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:34.880137 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.880390 26972 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.547s	user 0.000s	sys 0.002s
I20260709 12:14:34.889514 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:34.891752 26676 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:34.851971 26954 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:14:34.884294 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:34.895648 23391 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:34.894474 24013 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 12:14:34.894212 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 598) took 1335 ms (client timeout 1744 ms). Trace:
I20260709 12:14:34.870182 26992 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 12:14:34.902117 23387 rpcz_store.cc:269] 0709 12:14:33.558719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.611079 (+ 52360us) service_pool.cc:224] Handling call
0709 12:14:34.893857 (+1282778us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b5400190854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:34.894139 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1377661696}
I20260709 12:14:34.893868 26993 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:34.888247 26846 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.895637 26980 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:34.934917 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50715.7us
W20260709 12:14:34.922223 26782 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.022s	sys 0.022s
W20260709 12:14:34.921563 26847 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.022s	user 0.014s	sys 0.001s
I20260709 12:14:34.896603 26747 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:14:34.923733 26993 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:34.935441 26954 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:34.945214 26954 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:34.945698 26954 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:34.950189 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.951484 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:34.952250 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.953706 26900 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:34.954135 26900 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:34.954203 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.954578 26900 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:34.955053 23464 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:34.956018 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.956959 23461 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:34.957933 26993 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.958495 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.959249 26993 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:34.938488 26782 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.022s	sys 0.022s
I20260709 12:14:34.944746 26899 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:34.962514 26702 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:34.964933 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.953078 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:34.974869 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:34.939397 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.977896 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.990291 24014 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:14:34.991703 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52415.6us
I20260709 12:14:34.992420 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:34.992961 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.994655 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:34.995775 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:34.998658 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:34.999805 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.000139 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.000764 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.001390 23464 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:35.002096 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.003268 23464 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:35.003882 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.005151 23464 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:35.009421 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.010634 23463 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:34.914637 26951 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:34.983225 26847 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.060235 27008 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:35.063570 24011 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:14:35.063781 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:35.067783 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 61320.7us
I20260709 12:14:35.002214 26980 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:34.998139 26837 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:35.017189 26865 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.018496 26885 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:35.073544 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 71516.8us
I20260709 12:14:35.072597 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:34.940238 26950 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.016578 27003 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:35.081336 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.077147 26782 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:35.007484 26902 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:14:35.083603 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:34.942253 26887 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.075145 26980 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.012306 26983 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:14:35.016947 26820 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:14:34.998309 26832 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
W20260709 12:14:35.094028 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.081471 26865 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.107239 26832 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:35.107734 26832 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:35.108222 26865 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.085809 27003 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.109005 27003 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.108789 26832 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:35.102140 26995 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:14:35.094358 26450 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:35.089061 26875 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:35.071333 26847 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.125977 23387 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:14:35.085474 26837 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:35.150704 26837 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.088936 26782 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:35.097944 27007 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:35.117885 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 601) took 1539 ms. Trace:
I20260709 12:14:35.121579 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:35.099433 26980 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.164605 23396 rpcz_store.cc:276] 0709 12:14:33.578868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.069530 (+490662us) service_pool.cc:224] Handling call
0709 12:14:34.100689 (+ 31159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.128204 (+ 27515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.128238 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.788010 (+659772us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:34.788581 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.853039 (+ 64458us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:34.932223 (+ 79184us) log.cc:844] Serialized 14 byte log entry
0709 12:14:34.932873 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.932958 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.117516 (+184558us) raft_consensus.cc:1685] finished
0709 12:14:35.117665 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.117815 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27043,"prepare.queue_time_us":91193,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":27698,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":24075,"spinlock_wait_cycles":5841152,"thread_start_us":139506,"threads_started":2,"wal-append.queue_time_us":125455}
I20260709 12:14:35.133668 23320 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [6.8-8.9]
I20260709 12:14:35.134107 26875 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:35.135378 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:35.136442 26211 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9.
I20260709 12:14:35.136996 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:35.138314 27000 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:35.142674 26826 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:35.143596 26847 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.144347 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.120282 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:35.176337 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56149.9us
I20260709 12:14:35.158278 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.132215 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:35.169270 23388 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:35.173441 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.179858 23467 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:35.182066 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49992.6us
W20260709 12:14:35.182392 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.171342 26450 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.183475 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.187778 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.186738 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.181707 26846 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.174716 26924 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.212064 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.179728 26885 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:35.186000 26977 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:35.175333 26927 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:14:35.219744 26864 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:35.216456 26977 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.220453 26977 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:35.220645 24007 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:35.184528 26899 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:14:35.206332 26747 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:35.208500 26865 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.228865 26355 raft_consensus.cc:993] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:35.184415 26954 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.219763 26924 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:35.229485 23396 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:35.230906 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:35.231896 27002 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:14:35.232297 26927 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:35.232743 26862 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.181340 26987 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:14:35.248948 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:35.257234 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.215601 26885 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.175645 26900 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.280500 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:35.266472 26846 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.271235 23388 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.277325 23467 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.280046 26355 raft_consensus.cc:1081] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:35.249804 26988 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:35.208828 26450 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.225876 26826 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.286476 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:35.273018 26954 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:35.292568 26743 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.907s	user 0.009s	sys 0.002s
W20260709 12:14:35.292680 27010 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:14:35.282063 26885 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:35.258050 26818 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:14:35.308634 24008 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:35.279112 26943 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:14:35.311481 23388 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:35.314536 27007 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:35.308995 26743 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.329829 23468 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:35.332412 26450 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.287230 26846 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.318177 26744 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:35.318995 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:35.319531 26989 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:35.320883 23467 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:35.323487 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:35.271452 26875 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.327307 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:35.345930 23389 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:35.337096 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:35.337139 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:35.346308 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:35.326396 26976 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 12:14:35.346724 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.319557 26900 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:35.350101 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.354573 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.308180 26826 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.347999 26989 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:35.348951 26887 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:35.361477 26887 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.362013 26887 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.350445 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:35.359668 26954 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:35.359586 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.369045 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.355863 26976 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.370784 26989 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.374512 26989 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.374754 26976 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.353518 26450 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:35.380507 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:35.407241 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 56894.1us
I20260709 12:14:35.389114 23392 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:35.392104 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.413280 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.416584 23394 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:14:35.385543 27011 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:14:35.428517 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:35.380167 26988 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:35.380550 26951 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:35.389866 26450 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:35.439236 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:35.406055 26747 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:35.418668 26858 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:35.420696 26993 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.426380 26967 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.416296 26900 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:14:35.436925 26988 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.420639 27003 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.457804 26846 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:35.472532 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.467195 26988 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:35.452222 26747 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.496691 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.496719 26846 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.497550 24011 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:35.497771 26846 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.499878 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:35.500464 23461 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:35.501735 26670 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:35.502120 26670 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:35.459662 26993 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.506147 26993 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.461736 26967 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.510056 26826 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.503470 24015 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:35.453496 26890 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:14:35.520062 26893 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:35.520648 26893 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:35.521683 26893 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.522954 26893 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:35.533006 26825 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 12:14:35.535452 26898 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.535604 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.536206 23464 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:35.536013 26898 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.537519 27003 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.538292 27003 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:35.539777 26996 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:35.540194 26996 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:35.540012 26450 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.509872 23391 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:35.558943 27021 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:35.563503 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.565932 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:35.566543 24013 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
W20260709 12:14:35.569314 27018 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:14:35.541100 26996 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.574349 26996 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:35.575225 26211 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:35.575737 26211 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:35.578430 26670 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.579926 26670 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.614416 26894 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:35.617324 27003 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.576749 26211 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.622193 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.615078 26894 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:35.652050 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 179632us
I20260709 12:14:35.652819 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.657241 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.658733 23464 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:35.660429 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:35.666872 23395 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:35.662286 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:35.669787 23387 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.663362 23390 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:14:35.661407 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:35.668304 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:35.682010 23461 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:35.686450 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:35.687081 23467 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:35.687753 23391 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:14:35.690249 24007 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:35.691330 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.697850 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:35.700754 24016 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:35.704100 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:35.673597 26825 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:35.700516 26814 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:14:35.706966 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:35.683732 26826 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:35.685400 26872 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.678316 26549 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.709510 26946 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:14:35.712208 26825 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:35.701331 26211 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:35.716240 23320 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:14:35.639472 27003 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:35.750066 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.694871 26744 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.702116 26951 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:35.690069 26930 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:14:35.709998 26976 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:35.701018 26993 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.712093 26921 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:35.760406 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:35.793165 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.701524 26782 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.712419 26924 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 12:14:35.790380 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83486.6us
I20260709 12:14:35.747720 26846 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.721187 26826 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.790694 26747 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:35.747972 26549 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.773638 26976 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.741088 26872 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:35.821343 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 118582us
I20260709 12:14:35.760797 26744 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.821897 26846 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:35.854223 26747 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.854929 26865 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.007s
I20260709 12:14:35.855710 26898 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:35.881636 26885 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.556s	user 0.004s	sys 0.014s
I20260709 12:14:35.866389 27003 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:35.867300 26977 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.646s	user 0.008s	sys 0.020s
I20260709 12:14:35.808576 26826 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:35.868710 24011 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:35.867626 26872 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.875249 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:35.868213 26980 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.695s	user 0.019s	sys 0.010s
I20260709 12:14:35.801748 26950 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:14:35.854302 26954 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:35.861603 26976 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:35.898118 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.784286 26894 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.935138 23387 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:35.797698 26993 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:35.892385 26887 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.008s
I20260709 12:14:35.959077 26887 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.960526 26549 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.911933 23391 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:35.961119 26549 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.874285 26921 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:14:35.917698 26980 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.895112 26846 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.910959 27023 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:35.899993 26865 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:35.938876 27031 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:14:35.949620 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.950417 26872 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.969765 27032 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:14:35.964448 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:35.980767 24011 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:35.801093 26946 consensus_queue.cc:1243] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:14:35.905853 26885 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:35.938138 27027 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:14:35.898715 26747 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:35.978609 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:35.961865 23462 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:35.914100 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 115093us
I20260709 12:14:35.961362 26887 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:35.913582 26977 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:35.995744 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.969532 26894 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:35.940518 27003 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:14:36.007220 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:36.026154 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.033496 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.129492 26954 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:36.130630 26885 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.130918 26954 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.131606 26954 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:35.978156 26549 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.134757 26549 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.139140 26954 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.141402 26954 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.141857 26996 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.142411 26996 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.143253 27043 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:36.144521 23388 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:14:36.146838 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:36.149654 24011 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:35.985467 26825 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:36.151357 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:35.981774 26872 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.033901 26870 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:36.161029 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.161283 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.164934 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.162297 26870 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:36.166970 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:36.166726 26870 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.167512 24011 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:36.167749 26870 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:36.170059 26747 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:36.172545 26832 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.174561 24011 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:36.175295 26911 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:36.175833 26911 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:35.977859 27023 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:35.981211 26988 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.183074 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.151897 27041 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:36.162788 26450 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:36.191012 26747 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.191874 27041 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.192384 27041 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.196302 26826 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.199182 26846 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.200125 26826 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.186491 26996 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.176887 26911 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.200877 26872 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.202510 26977 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.203301 26988 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.181448 26924 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:14:36.191588 26747 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.198308 27041 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.198237 26980 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:36.208369 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.212103 26976 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.214323 27023 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:36.215445 26989 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.840s	user 0.014s	sys 0.018s
I20260709 12:14:36.217257 26747 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.222373 26450 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:36.223403 26450 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.238587 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 829) took 1427 ms. Trace:
I20260709 12:14:36.239019 24008 rpcz_store.cc:276] 0709 12:14:34.811521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.811668 (+   147us) service_pool.cc:224] Handling call
0709 12:14:36.238406 (+1426738us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b5400650ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:36.238504 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805122816}
I20260709 12:14:36.242022 26993 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.242575 26993 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.243008 26993 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.243875 26988 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.244385 26988 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.244896 26993 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.221817 26825 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.246376 26988 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.246845 26988 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.247429 26820 raft_consensus.cc:993] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:36.247252 26988 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.247972 26820 raft_consensus.cc:1081] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:14:36.249352 23316 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:36.250715 23319 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:36.251847 23317 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:36.256855 23318 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:36.263299 23319 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:36.268087 23315 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:36.268827 24009 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:36.275969 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:14:36.278401 23316 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:36.278880 23316 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:36.253044 26846 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:36.287349 27034 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:36.296772 26909 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:36.311031 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.308954 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:36.315235 23319 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:36.297919 26967 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:36.318527 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:36.319368 23313 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:14:36.319548 23321 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:36.319754 23313 pending_rounds.cc:85] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:14:36.320070 23313 pending_rounds.cc:107] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:14:36.320302 23313 raft_consensus.cc:2889] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:36.328310 23316 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:36.332834 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.273403 27044 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.352298 23319 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:36.352582 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:36.354475 23313 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:36.355542 23316 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.355645 23321 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:36.309921 27034 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:36.310163 26909 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:36.360292 23315 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:36.338109 26832 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.338172 26988 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.319188 26846 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:36.385524 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.388852 26909 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.392251 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.408289 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:36.411084 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.411725 23317 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.357276 27044 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.368582 26832 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.419310 26832 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.420190 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:36.435915 27030 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.677s	user 0.007s	sys 0.000s
I20260709 12:14:36.437952 26911 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:36.390177 27034 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.446448 27034 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.455900 26976 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.457484 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1669) took 1323 ms. Trace:
I20260709 12:14:36.465901 26976 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.466454 26976 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.468740 26846 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.471602 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 685) took 1080 ms. Trace:
I20260709 12:14:36.472004 23462 rpcz_store.cc:276] 0709 12:14:35.391673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.391797 (+   124us) service_pool.cc:224] Handling call
0709 12:14:36.471400 (+1079603us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54001072d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:36.471539 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267491200}
I20260709 12:14:36.461400 23393 rpcz_store.cc:276] 0709 12:14:35.134193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.153720 (+ 19527us) service_pool.cc:224] Handling call
0709 12:14:35.173289 (+ 19569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:35.175211 (+  1922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:35.175244 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:36.284572 (+1109328us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:36.284998 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 12:14:36.379032 (+ 94034us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:36.379070 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.379158 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:36.457005 (+ 77847us) raft_consensus.cc:1685] finished
0709 12:14:36.457035 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:36.457364 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":170,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":4416,"spinlock_wait_cycles":2557184,"thread_start_us":93672,"threads_started":1,"wal-append.queue_time_us":134375}
I20260709 12:14:36.480052 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.483711 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:60318
W20260709 12:14:36.484202 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.484305 23462 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:36.484827 23462 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:36.486472 23388 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:36.486624 24011 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:36.447813 27044 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:36.488075 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:36.488250 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.492347 24011 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.492910 23391 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:36.493799 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:36.490314 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:36.494325 23390 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:36.494557 23470 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:36.490658 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.497256 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.497932 23389 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:36.498327 23389 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:36.462862 26988 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.497974 23463 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:36.496734 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:36.462602 26450 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:36.481324 26747 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.486147 26967 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.508309 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:36.508980 23392 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:36.509102 23390 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:36.491945 26993 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.498092 26832 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.492261 27033 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.550467 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.520272 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:36.568331 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.509995 23389 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:36.513797 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:36.527294 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.557018 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:36.559413 23392 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:36.571907 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:36.573189 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.574572 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:36.503166 26846 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.575433 23392 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:36.569166 23462 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.579588 23461 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
W20260709 12:14:36.581568 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.580672 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:36.586052 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:36.587553 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.591120 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.591930 24015 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:36.510936 26992 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.596542 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:36.640413 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.652562 23392 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:36.652904 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:36.643939 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.654052 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.650008 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:36.650036 23462 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.656009 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.566409 26858 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:36.664898 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 168219us
I20260709 12:14:36.665841 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:36.677245 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:36.677888 24011 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:14:36.678222 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.678180 24011 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:36.504251 26988 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.554860 26967 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.681306 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:36.567166 26993 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.510607 26747 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.510478 26450 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.681505 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:36.574517 27033 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.568123 26832 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.662856 26925 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:36.697602 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.699020 26988 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.659860 26897 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:36.689735 27057 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:36.683736 24011 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.668035 26858 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.661593 26992 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.593130 26846 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:36.709115 26967 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.109555 26726 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:37.120350 26726 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.120998 26747 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:36.886049 26908 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:37.036733 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:37.122153 26908 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:37.122588 26726 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.038828 27049 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.115864 26897 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.123337 26908 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:36.793617 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.039047 26996 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:37.124846 26996 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.680544 26982 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:36.777757 23315 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:36.715706 23316 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:14:36.931000 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 833) took 1800 ms (client timeout 2118 ms). Trace:
W20260709 12:14:37.122668 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86061.6us
I20260709 12:14:37.141526 23388 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:37.141871 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.117512 26909 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:37.143808 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:37.144234 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.143615 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:37.125120 26897 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.819794 26450 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.685597 26820 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:36.491905 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:36.699275 26977 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:36.909247 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.136488 24011 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:36.955199 27058 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.900768 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.884407 26993 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.123648 27049 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.143549 23395 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:37.170493 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:37.151086 24009 rpcz_store.cc:269] 0709 12:14:35.130287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.130439 (+   152us) service_pool.cc:224] Handling call
0709 12:14:36.930779 (+1800340us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400780d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:36.930924 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506104704}
I20260709 12:14:37.152985 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:36.856745 26900 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.172559 24015 raft_consensus.cc:2410] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:14:36.915875 27023 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:37.039568 26980 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.954895 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.159516 27037 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:14:37.185135 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 230365us
I20260709 12:14:37.185365 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.186079 24007 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:37.186187 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.186821 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.164732 27039 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.156661 26996 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.187779 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.188172 23469 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:14:37.185155 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:37.188829 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.158015 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.189670 23464 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:37.160355 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:36.868567 26743 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:36.765587 26925 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:36.711422 27033 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:36.860457 26744 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.159073 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:37.168421 26820 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.184422 26900 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.197760 26864 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.198256 26858 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.201536 23321 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:37.201742 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.188478 27023 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.203163 27049 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.168668 26977 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.204408 27023 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.205660 26977 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.171974 26897 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:36.884932 27059 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:36.835248 26832 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:37.131402 27057 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:37.164774 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:37.167464 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.216821 23316 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:37.182782 27060 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.083707 23389 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:37.169754 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.174093 24011 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:37.177758 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.178606 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:36.763616 23360 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.050s (SENT) [suppressed 28 similar messages]
I20260709 12:14:37.225736 23461 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:37.179222 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.227358 23466 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:14:37.179718 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.025s	sys 0.026s
W20260709 12:14:36.838855 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.183003 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:37.228837 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.183863 24010 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 12:14:37.228999 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:37.229713 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.162285 26826 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:37.187641 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.189703 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.195041 26976 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.190428 23320 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:37.028505 26983 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:37.179430 27058 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.201537 26992 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.188584 26980 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.207904 27039 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.039354 27041 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:37.193961 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.206881 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.169580 26846 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.239960 23461 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
W20260709 12:14:37.240844 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53270.1us
I20260709 12:14:37.253001 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.205197 27007 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.143905 23387 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:37.210038 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.240682 26897 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.241690 26983 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.249167 27039 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.249404 26702 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:37.263847 26983 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.180584 26355 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:37.243490 27058 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.247359 26980 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.264509 26355 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.211783 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.136159 26988 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:37.266309 26988 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:37.266685 26355 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.215235 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:37.267496 26982 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:37.168381 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.267913 23469 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:37.267944 26982 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:37.268782 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.218843 26900 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.268944 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.217305 27059 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.269606 26993 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:37.269814 24009 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:37.248410 27041 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.270207 26993 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:37.271085 27059 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.220794 23470 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:37.271135 26993 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.271826 27041 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.213806 26864 consensus_queue.cc:1059] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:14:37.272502 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:37.273090 26864 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:37.273273 23470 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:37.273553 26864 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:37.274183 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.274468 26864 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.275116 27007 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.220834 27057 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.275738 26864 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:14:37.228896 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 45977.1us
I20260709 12:14:37.223286 26832 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:14:37.276873 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.222683 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.277762 26925 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.188418 24012 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:37.278263 23464 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:14:37.229728 23462 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:14:37.228331 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.025s	sys 0.026s
W20260709 12:14:37.286600 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 620817us
I20260709 12:14:37.288652 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:37.231621 26826 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.232194 26744 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.303207 26744 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.253299 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.248943 23463 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:37.250669 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.241684 27033 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.305876 27033 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.308218 27033 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.267168 26988 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:37.269116 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 110109us
I20260709 12:14:37.309514 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:37.294515 26897 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:37.222100 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:37.311103 26976 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:37.311444 23465 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:14:37.222813 26820 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.188045 26643 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:14:37.232482 27060 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.283865 26992 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.285986 23469 raft_consensus.cc:2410] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:14:37.280126 23464 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:37.344394 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:37.292837 23389 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:37.293835 23467 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:37.275696 23470 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:14:37.261595 27065 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.298220 26826 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.311100 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:37.379266 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68256.2us
I20260709 12:14:37.379961 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:37.295531 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:37.381517 26889 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 12:14:37.381819 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86357.7us
I20260709 12:14:37.383764 27060 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.385588 27044 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.386276 27044 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:37.386742 27044 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:37.387609 27044 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.316407 23461 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:37.291189 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86514.1us
I20260709 12:14:37.347548 23470 pending_rounds.cc:85] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 5
I20260709 12:14:37.389894 23470 pending_rounds.cc:107] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:14:37.390158 23470 raft_consensus.cc:2889] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.362107 26888 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:37.312892 26887 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:14:37.370316 23464 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:37.370687 27067 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.371134 26846 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.375536 26744 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.335335 27033 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:14:37.312285 26782 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:37.377544 26992 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:37.413882 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.360693 27058 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.392748 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.381366 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:37.427165 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.408176 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.408656 23465 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 3.
I20260709 12:14:37.425022 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.431347 24012 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:14:37.395519 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 142255us
W20260709 12:14:37.427718 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46411.6us
I20260709 12:14:37.434476 23389 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:37.434823 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.435429 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
W20260709 12:14:37.411234 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 647636us
I20260709 12:14:37.434964 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.410570 27060 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.394791 26897 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:37.408056 26820 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.454813 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.454865 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.457422 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.458516 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.459368 23461 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:37.413133 26888 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:37.428710 26925 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.378284 27065 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.405045 26643 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:14:37.465461 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.470229 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.467523 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.466686 23465 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:37.468838 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.496155 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.438380 27037 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:37.437084 27003 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:37.413367 26900 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:14:37.507105 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.512917 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.510442 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.515384 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.520707 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.550268 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.551709 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.552947 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.512562 23470 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:37.425622 26846 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.412896 27067 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.527441 23321 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:14:37.570413 23321 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:37.510576 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:37.528115 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:37.571465 24007 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:37.457456 26897 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:37.526963 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.544955 27065 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.460420 27063 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:14:37.545605 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.579701 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.580444 23316 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:37.491611 26983 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:14:37.547574 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.541703 24016 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:37.516726 23463 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:14:37.437624 27058 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.554490 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.585834 23463 pending_rounds.cc:85] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 4
I20260709 12:14:37.586109 23463 pending_rounds.cc:107] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:14:37.586138 23470 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:37.586493 23463 raft_consensus.cc:2889] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.587400 23470 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:37.457500 27007 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:14:37.555760 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.557766 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.470876 26820 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.571535 23312 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:37.590101 23318 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:37.528398 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.591003 23317 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:37.572769 23319 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:37.573827 23321 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:37.577282 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.506949 26925 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.545903 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.556402 23387 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:37.582464 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:37.464082 27060 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.583796 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.575619 26897 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:37.588344 23320 raft_consensus.cc:2393] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:37.530203 27003 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:37.511350 26702 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 12:14:37.599464 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:37.602794 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.614579 23469 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:37.611999 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.614584 23316 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:37.608101 23312 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:37.546898 26908 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.576556 26993 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.622951 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:37.613265 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 151474us
I20260709 12:14:37.624048 23314 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:37.613356 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 118395us
I20260709 12:14:37.591239 26976 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.625574 26908 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.629405 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.585449 27070 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.613245 26889 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:37.625301 26888 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.626372 27065 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.665187 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:37.674796 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:37.676662 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 176101us
I20260709 12:14:37.679109 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:37.680024 23387 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:37.612676 26864 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:14:37.682878 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.686585 23312 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:37.683651 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.688674 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.692641 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.697499 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.691679 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.707665 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.692482 27034 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:37.713618 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54877.7us
I20260709 12:14:37.716092 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:37.635198 26976 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:37.715960 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 279218us
I20260709 12:14:37.628086 26993 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.692039 27063 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:37.723783 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98431.5us
I20260709 12:14:37.693570 27077 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:37.724783 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.752095 27077 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:37.752909 27074 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:14:37.753620 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.676391 26889 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.712908 26992 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.755435 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.698720 26924 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:14:37.668148 27082 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:14:37.712263 26980 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:37.664187 27065 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.725091 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.658979 27070 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.690193 26988 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:14:37.767634 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.754336 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.769152 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.769476 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.754335 23389 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:37.756589 23316 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:37.771207 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:37.758350 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:37.760408 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.761833 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.724298 27043 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:37.753036 27077 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.774052 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.778723 23394 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:37.778898 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.774384 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.753755 27044 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:37.775437 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.768258 23395 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:37.779759 23390 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:37.782927 23388 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:37.784093 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:37.788293 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.724574 27072 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:14:37.793820 23462 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:14:37.718420 27003 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.780576 26996 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:37.780032 27049 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:37.761482 27063 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:37.789055 27082 consensus_queue.cc:1243] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:37.812098 26643 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:37.813030 26643 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:37.813517 26820 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.797582 27043 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:37.815126 26889 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.815680 26889 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.816105 26889 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:37.818307 27076 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:14:37.820087 26826 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:37.796459 26982 consensus_meta.cc:287] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.000s
I20260709 12:14:37.789958 26992 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:37.804442 27086 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:14:37.765384 27090 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:14:37.782213 27080 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.789994 26980 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:37.842216 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.798512 26900 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:37.844426 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.821538 26702 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:37.855055 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:37.800781 26846 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.860829 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.807824 27067 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.902040 23320 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:37.840936 27033 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:37.901575 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55509.4us
I20260709 12:14:37.841315 27043 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:37.907626 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 128464us
I20260709 12:14:37.908030 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:37.854110 26992 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:37.910043 23312 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:37.912818 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:37.808053 27081 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:14:37.859480 26820 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:37.857935 26925 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.862540 26702 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:37.915114 26837 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.853395 26982 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.916386 26992 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:37.937384 26870 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:37.932590 26837 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.914499 27033 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.933754 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:37.969985 26870 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.979043 24010 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:37.972813 27080 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:37.901473 26980 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.978287 26925 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:37.992053 27083 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:37.992172 26925 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:37.986476 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52839.6us
I20260709 12:14:37.987012 27078 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:37.987831 26870 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:37.930831 27070 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.974871 26889 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.930498 26820 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:37.915453 27043 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:37.997015 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.991786 26976 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:37.997681 26993 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.993412 26980 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:37.997874 27078 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:38.028273 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 244257us
W20260709 12:14:38.029265 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 106347us
I20260709 12:14:37.993145 26925 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:38.030803 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:38.034972 23396 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:38.035679 23396 raft_consensus.cc:614] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:38.029706 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.018630 26889 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:38.038350 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:38.037633 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 310264us
I20260709 12:14:38.025419 27065 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.030615 27078 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.044377 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:38.032773 26837 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:38.046783 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 288501us
I20260709 12:14:38.047397 27065 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:38.064966 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.049152 26977 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:38.040474 26980 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:14:38.069767 26980 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:38.070855 26980 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:38.072145 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.050024 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.073565 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.074256 24012 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:38.042236 26992 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:38.077230 26992 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.042387 26889 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.017607 26702 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.054266 26982 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.083247 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.054620 27043 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:38.056939 27088 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:14:38.043401 27067 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:38.058394 26820 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.046241 26993 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:14:38.063562 27063 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:38.048391 26925 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:38.093264 27067 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.100414 26889 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.100543 27067 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.100759 26908 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.064376 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.104085 23469 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:38.105028 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:38.054343 27033 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.105510 26702 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.105564 23469 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:38.091811 27065 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:38.108314 27065 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.081521 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:38.082819 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.084108 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.110466 23462 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:38.084489 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.111928 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.112522 23467 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:38.085110 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:38.086337 24008 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:38.086805 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.086936 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.088675 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:38.089972 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.076840 26897 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.091885 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.118148 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.118414 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.118685 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:38.095844 23396 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:38.090603 27034 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:38.098771 26909 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:38.113240 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.114384 24015 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:38.114656 23469 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:38.108700 27065 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.123298 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:38.127682 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.129372 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.130470 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.131145 23387 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:38.132933 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.133085 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.133953 23392 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:38.137106 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.127463 26993 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:38.131997 27043 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.138352 26976 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:38.138613 26993 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:38.138832 27043 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.140874 26993 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.144299 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:38.144824 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:38.145944 23390 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:38.146906 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.090597 26977 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:14:38.148515 26943 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:38.148890 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:38.125833 26782 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.003s
I20260709 12:14:38.149616 24008 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:38.150437 23468 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:38.148980 26943 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:38.141113 27043 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.151444 23387 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:38.158594 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.160230 24008 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:38.160835 24007 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:38.128080 26909 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:38.129721 26908 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:38.166396 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.208307 23392 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:38.157835 26943 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:38.209848 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.210526 24015 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:38.211354 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.211562 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.211903 24015 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:38.212635 23982 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.212227 26909 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.209852 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.213702 26909 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.213884 24008 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:38.214746 27072 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:38.215130 27072 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.215222 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.215488 27072 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:38.216928 26977 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:38.217063 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:38.217356 26977 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.217726 26977 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.217913 27039 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:38.218293 27039 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.218626 27039 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.220933 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.223320 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.221752 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:38.227114 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.231386 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.232236 24014 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:38.233740 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.234572 24007 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:38.232174 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.241719 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.243827 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.261283 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:38.263762 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 140544us
I20260709 12:14:38.152434 27035 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.264544 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:38.268028 23396 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:38.243739 27070 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:38.240595 26864 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:38.255966 26908 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:38.260059 26888 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:38.241009 27104 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.275832 27070 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:38.276837 26897 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.265945 27061 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:38.282212 27036 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.034s	sys 0.040s
I20260709 12:14:38.290483 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:38.292912 27070 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.294030 26782 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:38.294761 27067 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:38.283004 26888 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.282162 26864 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.282953 26908 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.286048 27061 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.299244 26782 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:38.298813 27036 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.034s	sys 0.040s
I20260709 12:14:38.301116 27104 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.304535 26888 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.306342 26864 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:38.322242 27077 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:38.323127 24013 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:38.285214 27035 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.310714 27061 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:38.327337 27033 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:38.328202 27060 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:38.332113 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 70911.4us
I20260709 12:14:38.307946 26908 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:38.324815 27065 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.338582 24013 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:38.348982 23467 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:38.349104 24013 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.350689 24013 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.352234 27065 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:38.339272 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74815.1us
I20260709 12:14:38.342554 24008 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:38.331672 26782 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:38.332901 27086 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:38.357355 27086 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:38.358233 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:38.359732 24013 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:14:38.367484 24013 pending_rounds.cc:85] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 7
I20260709 12:14:38.369601 24013 pending_rounds.cc:107] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:14:38.369926 24013 raft_consensus.cc:2889] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:38.368901 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.372203 26978 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:38.372648 26978 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:38.373474 26978 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.374706 26978 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:38.377700 27060 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:38.380065 27058 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:38.332394 27112 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:38.344861 27033 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.329885 27078 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.383174 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.333305 26943 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.386898 27065 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:38.391855 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.391368 27065 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.388460 27112 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:38.358260 27086 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.410312 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:38.395363 27073 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.972s	user 0.004s	sys 0.000s
I20260709 12:14:38.411832 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2070) took 1179 ms. Trace:
I20260709 12:14:38.390009 27033 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:38.394157 23464 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:14:38.434145 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:38.470000 24016 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:38.391237 27092 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 12:14:38.420083 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.446651 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.438560 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.477346 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:38.478335 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.433607 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:38.436110 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:38.435631 23319 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:38.428270 23470 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:38.474529 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.478647 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.436108 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:38.394122 27065 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.438231 23462 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:14:38.510551 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 74541.5us
I20260709 12:14:38.511602 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:38.479578 23318 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:38.514159 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:38.517683 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:38.517784 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:38.518581 23314 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:38.410436 27115 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:38.522217 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.400368 27112 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.401273 27086 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:38.426780 23463 rpcz_store.cc:276] 0709 12:14:37.232594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.305289 (+ 72695us) service_pool.cc:224] Handling call
0709 12:14:37.307610 (+  2321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.586931 (+279321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.411592 (+824661us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:38.411770 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479902464}
I20260709 12:14:38.434085 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:38.529901 23316 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:38.518720 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:38.480080 23317 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:38.411013 27070 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.411689 26980 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:38.404402 26826 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.395160 27039 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.474361 27117 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:38.401702 26864 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.393676 26908 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.522432 27115 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.559370 27115 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:38.561839 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:38.526829 27058 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.557404 27067 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.480568 26977 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.523007 27034 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 12:14:38.679229 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.572350 23317 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:38.595309 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:38.576006 27104 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.818596 27058 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.744490 27023 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:38.819754 27023 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.542447 27070 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.821508 27058 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.528904 27108 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:38.825119 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 300002us
I20260709 12:14:38.822058 27023 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.639133 24012 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:38.578372 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:38.652001 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2383) took 3283 ms (client timeout 3027 ms). Trace:
W20260709 12:14:38.831528 23468 rpcz_store.cc:269] 0709 12:14:35.348646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.349850 (+  1204us) service_pool.cc:224] Handling call
0709 12:14:38.631794 (+3281944us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b54003ee4d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:38.631951 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6430650624}
I20260709 12:14:38.826208 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:38.828125 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 662) took 1319 ms. Trace:
I20260709 12:14:38.793371 26943 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:38.832652 24010 rpcz_store.cc:276] 0709 12:14:37.509181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.509300 (+   119us) service_pool.cc:224] Handling call
0709 12:14:37.533512 (+ 24212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:37.980428 (+446916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.827856 (+847428us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54005613d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:38.828040 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024039296}
W20260709 12:14:38.792179 27072 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.000s
W20260709 12:14:38.743499 27069 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.319s	user 0.005s	sys 0.000s
W20260709 12:14:38.742817 27118 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:14:38.909539 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 83432.2us
I20260709 12:14:38.910420 23461 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:38.912086 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:38.566361 27114 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 12:14:38.904873 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.789184 27060 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:14:38.824607 27116 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.829020 27108 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:38.992635 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:38.994485 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.996316 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:38.834107 27058 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:38.555569 27117 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:38.764112 27078 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.555495 26980 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.018105 24011 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:39.019661 26980 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.021479 27070 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.996734 27108 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:39.022040 26864 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:38.816428 26993 raft_consensus.cc:2764] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:38.979714 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:38.980422 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:38.562958 27039 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.031077 27058 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.032164 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.010175 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.012255 24010 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:14:39.034160 27086 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.501s	user 0.003s	sys 0.006s
I20260709 12:14:39.034533 24008 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:39.035907 26888 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
I20260709 12:14:38.833521 26909 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.163s
I20260709 12:14:38.968008 27067 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:38.896761 26782 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.039814 27115 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.040287 26782 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.041029 23466 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:39.010164 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.017491 27117 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.031422 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.047567 23470 raft_consensus.cc:2410] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:14:39.032032 27039 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:38.808506 27061 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.036360 26888 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:38.910099 27072 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.008122 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2082) took 1763 ms. Trace:
I20260709 12:14:39.081532 23464 rpcz_store.cc:276] 0709 12:14:37.244962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.410959 (+165997us) service_pool.cc:224] Handling call
0709 12:14:37.433240 (+ 22281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:38.470396 (+1037156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.995617 (+525221us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b54001072d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:39.008055 (+ 12438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209352448}
I20260709 12:14:38.812507 27065 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.086670 23469 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:39.051141 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 139167us
I20260709 12:14:38.996939 26977 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.088701 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.060140 27034 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.999243 27116 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.065644 27058 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.066537 26993 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:39.070472 27115 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.072389 27070 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:39.074383 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 1.02219e+06us
I20260709 12:14:39.074520 27109 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:39.105088 27034 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.075241 27106 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.118939 27065 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.118868 27106 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.119750 27034 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.075340 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:39.075827 27107 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.121502 26888 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.121798 27107 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.122756 27072 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.125154 27108 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.125530 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.126307 23466 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.127265 27108 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.115422 27070 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:39.127799 26993 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.128726 27070 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.129905 26993 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:39.132033 27033 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.619s	user 0.003s	sys 0.006s
I20260709 12:14:39.132555 27033 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.134521 26980 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:39.134770 27061 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.134982 26980 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.135388 26980 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.079622 26908 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.080523 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:39.083964 24008 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:39.136456 23391 raft_consensus.cc:606] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:39.137265 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 135544us
W20260709 12:14:39.097645 23462 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.003s	sys 0.012s
W20260709 12:14:39.098469 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.102231 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:39.049048 26864 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:39.081332 27003 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.166189 26864 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.168056 27126 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:39.168429 26864 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.168874 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.169023 27034 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:39.169327 27128 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:39.169855 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.170286 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 846) took 1085 ms. Trace:
I20260709 12:14:39.170389 27034 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:39.170689 24012 rpcz_store.cc:276] 0709 12:14:38.084780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.084897 (+   117us) service_pool.cc:224] Handling call
0709 12:14:39.170143 (+1085246us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54007f0154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:39.170238 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995529472}
I20260709 12:14:39.171455 27034 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:39.172025 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.172353 27126 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:39.172935 27126 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:39.173128 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.173746 27108 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.174046 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.174335 27108 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:39.173794 27126 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.174762 27108 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.175261 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.176070 23464 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:39.177117 23464 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:14:39.177515 23461 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:39.178974 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.179564 24012 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:39.180110 27119 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.703s	user 0.004s	sys 0.019s
I20260709 12:14:39.180685 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.181854 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.183844 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.185250 27033 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.186878 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.188256 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.142313 24008 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:39.143256 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.143767 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.191847 24013 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:39.144701 24010 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:39.145516 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.145741 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.195667 27070 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:39.173817 23387 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:39.197793 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.198493 24008 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.200074 24007 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.148612 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.149693 23470 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:39.150590 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.150715 23467 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:39.151257 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.203655 23463 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:39.153328 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.204541 23318 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:39.154560 27061 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:39.155658 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.156076 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.205745 24012 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:14:39.206295 23314 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:39.155717 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.207401 23313 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:39.158591 26993 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:39.159479 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:39.209026 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49631.3us
I20260709 12:14:39.210140 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:39.161768 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86524.7us
I20260709 12:14:39.162097 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:39.211019 23391 raft_consensus.cc:606] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:39.162472 26897 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.164674 27117 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.169515 23469 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:39.116915 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.213699 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:39.214051 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.193931 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:39.194204 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 342) took 19987 ms (client timeout 15000 ms). Trace:
I20260709 12:14:39.194581 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.215121 23317 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.146114 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.217375 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.217875 23466 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.218138 24013 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.218289 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.204991 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:39.139178 23462 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.004s	sys 0.012s
I20260709 12:14:39.160285 27036 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:39.227046 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:39.227123 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:39.229607 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.230317 23469 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13.
I20260709 12:14:39.230401 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.231112 24007 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.231860 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.232713 24013 raft_consensus.cc:2368] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13.
I20260709 12:14:39.233924 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.237084 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.241089 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:39.211549 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63012.8us
I20260709 12:14:39.222230 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.214977 23315 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
W20260709 12:14:39.183874 27112 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.641s	user 0.008s	sys 0.001s
I20260709 12:14:39.212478 26897 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.252435 27109 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.252842 27109 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.252270 23214 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.253199 27109 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.254104 27117 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.253960 23214 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.254874 27081 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:39.256072 23214 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.257051 27081 raft_consensus.cc:993] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:39.172353 27034 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:39.252437 26897 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.257635 27034 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:39.258248 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:39.214911 23217 rpcz_store.cc:269] 0709 12:14:19.206923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.221948 (+ 15025us) service_pool.cc:224] Handling call
0709 12:14:39.194100 (+19972152us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:32.066045 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:32.091513 (+ 25468us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:32.091580 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:14:32.091608 (+    28us) tablet.cc:662] Decoding operations
0709 12:14:32.255135 (+163527us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:32.291486 (+ 36351us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:32.294002 (+  2516us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:14:32.591877 (+297875us) tablet.cc:701] Row locks acquired
0709 12:14:32.612858 (+ 20981us) write_op.cc:260] PREPARE: finished
0709 12:14:32.650329 (+ 37471us) write_op.cc:270] Start()
0709 12:14:32.668799 (+ 18470us) write_op.cc:276] Timestamp: P: 1783599272636005 usec, L: 0
0709 12:14:32.703235 (+ 34436us) op_driver.cc:348] REPLICATION: starting
0709 12:14:32.759305 (+ 56070us) log.cc:844] Serialized 5638 byte log entry
0709 12:14:33.937246 (+1177941us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:34.701683 (+764437us) op_driver.cc:464] REPLICATION: finished
0709 12:14:34.804557 (+102874us) write_op.cc:301] APPLY: starting
0709 12:14:34.817768 (+ 13211us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:34.860059 (+ 42291us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:34.860109 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:34.865684 (+  5575us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:34.866036 (+   352us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:14:34.866081 (+    45us) write_op.cc:312] APPLY: finished
0709 12:14:34.866516 (+   435us) log.cc:844] Serialized 101 byte log entry
0709 12:14:34.866704 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:34.866849 (+   145us) write_op.cc:454] Released schema lock
0709 12:14:34.866912 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:34.922894 (+ 55982us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:34.931059 (+  8165us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":276452,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":62165,"apply.run_cpu_time_us":4673,"apply.run_wall_time_us":110615,"num_ops":14,"prepare.queue_time_us":579173,"prepare.run_cpu_time_us":28918,"prepare.run_wall_time_us":866971,"raft.queue_time_us":370991,"raft.run_cpu_time_us":50307,"raft.run_wall_time_us":123789,"replication_time_us":1973297,"spinlock_wait_cycles":116315520,"thread_start_us":684878,"threads_started":4,"wal-append.queue_time_us":167125,"wal-append.run_cpu_time_us":36471,"wal-append.run_wall_time_us":2940125}]]}
I20260709 12:14:39.252140 27101 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.217181 26826 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.262006 27101 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.262486 27101 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.262633 26826 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:39.263114 26826 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.263463 26826 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.261348 27049 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:39.264380 23465 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:39.264657 27081 raft_consensus.cc:1081] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:39.197387 27003 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.104127 27043 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 12:14:39.234645 23466 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:39.258564 27034 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.199930 27128 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.218853 26908 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.123903 27107 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:39.274120 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.244917 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.283982 23214 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.284828 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.249534 27067 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:39.213351 26993 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:39.271073 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49110.5us
I20260709 12:14:39.121124 27106 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.266782 27115 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:39.278069 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:39.300328 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.291901 23318 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:39.292302 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.272750 23317 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:39.292984 23214 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 3 to 5, term changed from 1 to 6, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.302981 23317 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:14:39.294523 23315 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:39.262506 27099 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:39.267776 26864 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.305625 26864 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.308537 26864 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.275609 26978 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.283435 26908 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.283171 27080 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.318629 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.275547 27097 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.319847 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.311583 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.317940 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.318470 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.273964 26702 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:14:39.294087 27036 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:39.357822 23313 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:39.358975 24013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:14:39.359148 23314 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:39.322284 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95260.6us
I20260709 12:14:39.388662 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.300498 26993 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.294025 26889 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:39.402559 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.294286 27067 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:39.324842 26978 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.318745 26888 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:39.356839 27080 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.412379 27080 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.364166 26897 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.269873 27112 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.414547 27112 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.414966 26897 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.365218 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.366160 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 3 to 6, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:39.357249 27097 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.356815 26643 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.376400 27092 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.379796 24013 raft_consensus.cc:606] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:39.402985 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.358106 27099 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:14:39.401081 23315 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.409854 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.428311 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.412411 26978 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.371178 27108 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.319556 27058 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:39.318940 27034 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.420114 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.420621 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:39.466233 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 148417us
I20260709 12:14:39.466471 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.466787 23394 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:39.320626 26864 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.323781 26908 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.321571 27113 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.431154 23362 leader_election.cc:400] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13.
I20260709 12:14:39.482731 23362 leader_election.cc:403] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:14:39.482970 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.351666 27106 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.434504 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.427476 27097 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.378731 27115 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:39.433838 27003 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:39.405210 27036 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.465308 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.467407 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.510121 23390 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:39.511543 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.465971 23391 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:39.408386 27067 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:39.474660 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.517510 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.423755 27128 raft_consensus.cc:2764] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:14:39.483546 23394 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.484330 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:39.483258 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52307.4us
I20260709 12:14:39.424140 26911 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.433790 23214 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.488304 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.490098 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.489960 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.310770 26980 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.493696 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.494695 23321 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:39.434084 26643 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.481525 26888 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:39.495697 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.496527 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.528955 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:39.418231 27081 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 12:14:39.532109 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43886.6us
I20260709 12:14:39.533145 23394 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
W20260709 12:14:39.500520 27133 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:14:39.483736 26864 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.435583 27049 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:14:39.433686 26897 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.504853 23463 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:39.465623 27058 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.507257 27035 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.433902 27057 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:39.509320 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.418650 26889 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.481256 27033 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:39.535923 26702 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:39.530488 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.497335 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.573261 23317 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:39.533913 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.536149 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.574097 23394 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:39.544389 27130 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:39.519991 27036 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:39.481760 27080 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:14:39.556594 27112 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.546197 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.548702 23387 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:39.588133 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.524137 26978 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:39.550010 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.550748 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.523890 27043 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.523591 26925 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.560387 23320 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.560287 23214 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.565384 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.484011 27127 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:14:39.463198 27092 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.494915 27134 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.583515 27082 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:39.574918 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.542120 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.577322 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.619720 23980 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.580880 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2386) took 1309 ms. Trace:
I20260709 12:14:39.620757 23396 rpcz_store.cc:276] 0709 12:14:38.271224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.273086 (+  1862us) service_pool.cc:224] Handling call
0709 12:14:38.280623 (+  7537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:39.076378 (+795755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:39.076412 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:39.076898 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 12:14:39.293466 (+216568us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:39.293502 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.293576 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:39.580614 (+287038us) raft_consensus.cc:1685] finished
0709 12:14:39.580644 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.580806 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":795334,"raft.queue_time_us":796416,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"thread_start_us":216269,"threads_started":1,"wal-append.queue_time_us":361733}
I20260709 12:14:39.548102 26897 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:39.582779 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52386.3us
I20260709 12:14:39.558846 27007 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.508772 27106 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.526425 26911 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.590034 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.548843 23469 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:39.595672 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.572479 27035 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:39.602807 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56822.2us
I20260709 12:14:39.645640 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.648463 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:39.606441 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2419) took 1135 ms. Trace:
I20260709 12:14:39.599483 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:39.631331 26897 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.630507 27104 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.649469 23462 rpcz_store.cc:276] 0709 12:14:38.471028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.513744 (+ 42716us) service_pool.cc:224] Handling call
0709 12:14:38.514024 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:39.221610 (+707586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:39.221699 (+    89us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:39.222260 (+   561us) log.cc:844] Serialized 52 byte log entry
0709 12:14:39.431370 (+209110us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:39.431414 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.431491 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:39.606159 (+174668us) raft_consensus.cc:1685] finished
0709 12:14:39.606190 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.606370 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20922,"raft.queue_time_us":229738,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":851437,"threads_started":2,"wal-append.queue_time_us":306977}
I20260709 12:14:39.559938 27058 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.604740 26909 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:39.605031 27061 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.709249 27061 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.711359 27061 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.585590 27116 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.596899 27043 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.712553 27043 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.610958 27077 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.713816 26909 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:39.713555 27077 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.714268 26909 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.715720 27077 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.613149 26702 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.586905 27086 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:39.578898 27130 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:39.629328 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.723698 23394 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:39.630125 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.570179 27057 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.725972 23320 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:39.726361 26911 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.726572 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.727146 23390 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
W20260709 12:14:39.633309 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91312.8us
I20260709 12:14:39.727169 26643 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.730595 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.733361 27081 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.734175 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.734911 23387 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:39.735232 27106 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.735399 26978 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13.
I20260709 12:14:39.735656 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.744043 23387 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:39.744489 26908 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.607959 23391 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
W20260709 12:14:39.745641 27132 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:14:39.643029 27107 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.652689 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.754419 23396 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:39.673246 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.673868 23362 leader_election.cc:400] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13.
W20260709 12:14:39.755695 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 81939.1us
I20260709 12:14:39.673928 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.642310 27007 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.688603 26889 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.598380 27112 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.608459 26925 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.669160 27063 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:39.703053 27092 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.706239 27082 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:39.590673 27115 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.822449 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.706154 23214 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:39.610347 26888 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.644706 26782 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:39.725582 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.583494 27033 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:39.727988 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.650307 27035 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:39.633734 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.841699 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.669319 27117 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.846524 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.847163 23390 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:39.848012 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.748160 23391 pending_rounds.cc:85] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 7
I20260709 12:14:39.849608 23391 pending_rounds.cc:107] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:14:39.849879 23391 raft_consensus.cc:2889] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:39.850629 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.851027 26908 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.851389 23396 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 9 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:39.667526 23388 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:39.713127 26993 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.855150 26993 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.855041 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.751207 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:39.856419 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.856514 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.856607 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.857146 23467 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:39.857482 23468 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.717304 26702 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.858614 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.722077 27108 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.859114 23467 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:14:39.746805 27130 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:39.859925 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.745760 26980 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.723176 27086 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:39.861874 27086 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:39.862552 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:39.864621 23319 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:39.844887 27033 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.754699 23393 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:14:39.799249 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 179606us
I20260709 12:14:39.866816 27033 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:39.867632 27138 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:14:39.867978 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.849099 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.864625 27035 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.809693 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:39.869771 23390 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:14:39.869963 24012 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.870553 27077 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:39.871555 27077 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:39.815161 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 161505us
I20260709 12:14:39.871948 23319 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:14:39.834479 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.872607 27092 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:39.873014 27092 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:39.855041 23316 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:39.875651 23463 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:14:39.877449 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.836794 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.878770 23394 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:39.880569 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:39.812901 27109 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.001s
I20260709 12:14:39.880718 27130 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.844456 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.845005 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.883519 26925 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.812261 27063 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.884048 27063 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:39.781571 27127 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:39.886076 27127 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.887115 27117 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:39.888180 27127 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.846089 23461 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
W20260709 12:14:39.756825 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 239461us
I20260709 12:14:39.808543 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:39.821533 27101 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.000s
I20260709 12:14:39.890436 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.890918 27101 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.891813 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.893218 23319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:14:39.893792 23319 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:39.893644 26980 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:39.894300 23319 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.894361 27101 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.877650 27092 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:39.895315 27136 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.389s	user 0.004s	sys 0.009s
I20260709 12:14:39.895710 27092 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:39.881759 27109 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:39.896651 23395 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:39.897614 27109 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:39.899632 23321 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:14:39.853114 27112 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:39.900923 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56261.8us
I20260709 12:14:39.834851 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:39.829221 26826 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.001s
W20260709 12:14:39.854974 27141 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:14:39.898819 27007 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:39.907626 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.900624 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:39.914582 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.915697 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.919691 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.921293 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.937872 23466 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.923362 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.781324 26911 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:14:39.937366 24007 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.955919 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:39.893682 27108 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:39.943095 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:39.942405 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108911us
I20260709 12:14:39.945155 23395 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:39.962788 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:39.964665 23314 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:39.953914 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.881320 26864 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:39.954694 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:39.930285 27127 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:39.969400 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.983667 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:39.957830 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.958335 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.956949 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.984841 24016 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.984457 23313 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:39.985865 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.986056 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:39.986907 23313 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:39.987890 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:39.937779 27070 consensus_meta.cc:287] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.742s	user 0.011s	sys 0.000s
I20260709 12:14:39.993746 24015 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:39.956943 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:39.969017 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.969177 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:39.971274 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:39.995270 23470 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:39.974550 23387 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:39.990262 23467 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:14:39.996318 23469 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:39.938973 26643 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:39.991533 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.997792 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:39.995639 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:39.943334 26993 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:39.849261 26897 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.943465 26909 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:40.004997 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46782.6us
I20260709 12:14:40.009704 23470 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:39.967096 27106 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:39.930305 27095 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:14:39.982700 27142 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:40.022701 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:40.022930 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:39.936895 26826 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:39.983120 27127 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.954015 26889 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:14:40.028708 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:39.982674 23319 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.001953 27101 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.072190 27142 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.033744 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:40.076459 23319 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.004451 27140 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.006138 26782 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.016863 27126 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:40.005259 27003 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:40.078187 27147 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:40.078735 27142 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:40.078992 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45362.4us
I20260709 12:14:40.079541 23462 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:40.006973 27108 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:40.075749 23392 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:40.092077 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:40.014205 26993 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.025995 27113 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:40.092290 23392 raft_consensus.cc:614] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:39.966387 27134 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 12:14:40.014179 27150 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:39.997004 27043 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.099476 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:40.069020 26643 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.075002 27127 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.032035 27061 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:14:40.143844 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 96486us
I20260709 12:14:40.075968 27065 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.069273 26977 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:40.158854 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59468.2us
I20260709 12:14:40.101080 26993 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:40.160126 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:40.032083 27148 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:40.090436 27003 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:40.102020 26943 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:40.140630 27113 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.141660 27007 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:40.079939 27058 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.091110 27126 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:40.156469 27150 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:40.098523 27116 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:14:40.102226 27036 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.189179 26909 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.189307 27043 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.189688 27036 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.154218 27037 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:14:40.154563 26643 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.192550 26643 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:40.193253 27158 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:14:40.144098 27107 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:14:40.176064 27007 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:40.196458 27007 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:40.197830 27063 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.173991 27113 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.174387 27070 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.205032 24016 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:40.189591 26911 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:40.184794 26943 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:40.207278 27095 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:40.206959 26943 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.210150 26897 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.210408 26978 raft_consensus.cc:993] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:14:40.207671 27095 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:40.211329 27095 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.212519 27095 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.213739 26978 raft_consensus.cc:1081] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:14:40.213918 27067 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.004s
I20260709 12:14:40.205821 26911 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:40.214334 27067 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.215245 26911 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.216382 27067 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.216602 26911 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.220346 27086 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.155431 26782 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 12:14:40.224483 27157 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:14:40.170317 27162 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.187572 27126 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:40.228570 27130 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:40.229984 27152 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:14:40.142217 27108 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.234664 26826 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.169833 27003 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.236057 24014 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:40.161213 26977 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:40.154281 27112 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:14:40.096151 27144 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:14:40.233176 27108 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.185056 27150 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:40.096519 27140 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.246104 27057 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:40.236090 27113 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.269714 27166 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:40.273653 27166 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:40.275296 27113 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.274886 27166 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.276682 27166 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:40.278414 27086 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.281244 27140 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:40.265271 27097 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.005s
I20260709 12:14:40.282389 27097 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.291687 27126 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.306233 27127 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:40.347078 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 275837us
I20260709 12:14:40.351235 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:40.355063 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:40.357085 23315 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:40.359118 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.084021 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.073192 23318 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 4. Current term is 5. Ops: []
I20260709 12:14:40.361390 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.380693 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.362303 23317 raft_consensus.cc:2368] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
I20260709 12:14:40.171218 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:40.384157 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:40.384524 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 213248us
I20260709 12:14:40.384711 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.358904 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.388161 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.391258 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:40.361075 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.411360 27037 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.412568 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.412271 27037 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:40.416427 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.416602 27037 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.359233 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.420660 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.422226 27067 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:40.423354 27043 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:40.364241 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:40.426043 27063 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:40.365197 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 336456us
I20260709 12:14:40.427707 27081 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:40.428275 27081 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:40.428198 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:40.428949 27081 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:40.374590 23388 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:40.431052 26782 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:40.431993 26782 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:40.433719 27082 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:40.434492 27058 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:40.433982 26782 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.402067 26702 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.403380 26864 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:40.436404 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.404587 27023 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.246266 26977 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:40.396665 26826 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:40.404083 26888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.002s	sys 0.001s
I20260709 12:14:40.437132 26864 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:40.438069 27077 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.440855 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:40.285148 27097 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.422250 27080 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:40.446141 23312 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:14:40.392388 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:40.376466 23389 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:40.375254 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:40.424754 27067 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:14:40.463270 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 440392us
I20260709 12:14:40.364004 23313 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:40.447649 27134 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:40.471887 23321 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:40.442615 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:40.479429 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 887) took 1167 ms. Trace:
I20260709 12:14:40.480147 23319 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:40.478197 26864 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.483568 23360 leader_election.cc:400] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
I20260709 12:14:40.484249 27037 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:40.485082 27077 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.457484 27126 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.496558 27108 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:40.497008 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1715) took 1013 ms. Trace:
I20260709 12:14:40.497594 23389 rpcz_store.cc:276] 0709 12:14:39.483940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.484060 (+   120us) service_pool.cc:224] Handling call
0709 12:14:40.496694 (+1012634us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54002e9d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:40.496890 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966766464}
I20260709 12:14:40.497650 27023 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.499083 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:40.489476 26888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.003s	sys 0.001s
I20260709 12:14:40.503491 27077 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.478349 23316 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:40.493049 23360 leader_election.cc:403] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:14:40.505915 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 892) took 1195 ms. Trace:
I20260709 12:14:40.506369 23312 rpcz_store.cc:276] 0709 12:14:39.322529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.613082 (+290553us) service_pool.cc:224] Handling call
0709 12:14:40.445587 (+832505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:40.494292 (+ 48705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.495725 (+  1433us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400388e50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:40.505859 (+ 10134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913750272}
I20260709 12:14:40.491804 23314 rpcz_store.cc:276] 0709 12:14:39.308761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:39.606639 (+297878us) service_pool.cc:224] Handling call
0709 12:14:39.610594 (+  3955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:39.965138 (+354544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.393900 (+428762us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:40.475830 (+ 81930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":813820288}
W20260709 12:14:40.513394 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 121224us
I20260709 12:14:40.486574 27080 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:40.491334 27134 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:40.519265 26702 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:40.519582 27077 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:14:40.522505 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 147667us
W20260709 12:14:40.523983 24007 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.563s	user 0.015s	sys 0.001s
I20260709 12:14:40.523854 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2191) took 1020 ms. Trace:
I20260709 12:14:40.524379 23313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:14:40.508121 27023 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:40.532573 27080 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.534035 23320 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:40.537479 27161 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:40.558841 24007 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:40.552449 26826 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.560981 27080 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:40.561553 24010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.003s	sys 0.007s
I20260709 12:14:40.538234 27108 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:40.558045 23313 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:14:40.563090 23313 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:40.548309 23317 rpcz_store.cc:276] 0709 12:14:39.503062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.477092 (+974030us) service_pool.cc:224] Handling call
0709 12:14:40.523110 (+ 46018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.523472 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:40.523500 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:40.523562 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:40.523614 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.523669 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:40.523767 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:40.557226 23466 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.007s
I20260709 12:14:40.570364 27161 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:40.570855 27161 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:40.570952 27007 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.571972 27161 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:40.573885 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 131573us
W20260709 12:14:40.577669 24008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.002s	sys 0.019s
W20260709 12:14:40.578109 24008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.003s	sys 0.020s
I20260709 12:14:40.578245 27108 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:40.579013 23316 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:40.579704 23315 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:40.579586 27108 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:40.586338 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:40.587518 26977 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.588622 26925 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:14:40.581842 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69489.5us
W20260709 12:14:40.589851 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 106588us
I20260709 12:14:40.581722 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2193) took 1027 ms. Trace:
I20260709 12:14:40.547727 26702 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:40.594625 23320 rpcz_store.cc:276] 0709 12:14:39.538208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.492922 (+954714us) service_pool.cc:224] Handling call
0709 12:14:40.533775 (+ 40853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.565330 (+ 31555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:40.565519 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:40.548727 27134 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:40.581219 26864 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.548403 26888 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:40.595755 26925 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:40.590233 27081 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.562345 26826 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.582221 27007 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:40.608477 26925 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.606326 27070 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:40.601366 26864 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.603926 26993 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.652145 23465 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:40.598057 26702 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.656845 26925 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:40.601279 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:40.598405 23466 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:40.631132 27150 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.610702 27161 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.638675 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:40.642164 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:40.642546 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:40.585352 27147 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:40.629190 23313 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.599167 27165 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:40.643347 27101 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:14:40.637883 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:40.664951 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:40.666541 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:40.676401 23315 raft_consensus.cc:2368] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:14:40.676118 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:40.683128 23465 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:40.623265 27003 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:40.691859 23313 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.681116 27043 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:40.693421 27127 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.695220 26977 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.697357 27150 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.697641 26980 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:40.698542 23462 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:40.706209 27095 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:40.706705 27095 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:40.706944 23470 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 3.
I20260709 12:14:40.678084 26826 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:14:40.707675 27095 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:40.709227 26993 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:40.693496 27070 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.667075 27147 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:40.807122 23319 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:41.073572 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:40.723484 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:40.726229 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:40.931878 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.371420 27070 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.093593 27081 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:40.961635 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:40.827629 27007 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.725855 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:40.829258 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:40.790324 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:40.793601 23980 leader_election.cc:400] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
W20260709 12:14:41.365548 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.000s	sys 0.014s
I20260709 12:14:40.725554 23387 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:14:41.358229 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 699411us
I20260709 12:14:40.726965 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:40.724300 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.362933 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:41.000536 24008 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:41.380461 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2195) took 1632 ms. Trace:
I20260709 12:14:40.726411 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:41.358299 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 715825us
I20260709 12:14:41.358505 27095 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:40.710009 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.412108 23980 leader_election.cc:403] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:14:40.807420 27036 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:41.386257 23395 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:41.380193 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:41.415442 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.001s	sys 0.018s
I20260709 12:14:41.414345 23319 rpcz_store.cc:276] 0709 12:14:39.748287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.505719 (+757432us) service_pool.cc:224] Handling call
0709 12:14:40.537207 (+ 31488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:41.370865 (+833658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.380150 (+  9285us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400295254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:41.380389 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620120448}
I20260709 12:14:40.946740 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:40.932736 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:40.916297 27003 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.364740 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:40.965991 27177 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.483s	user 0.000s	sys 0.002s
I20260709 12:14:40.998669 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:41.418907 27067 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:41.416661 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:41.429849 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 738106us
W20260709 12:14:41.413997 27165 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.000s
I20260709 12:14:41.424902 27036 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.425534 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.425292 24016 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:41.425225 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62407.4us
I20260709 12:14:41.383064 26782 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:41.424968 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:41.425612 24014 raft_consensus.cc:2376] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12. Re-sending same reply.
I20260709 12:14:41.358915 27108 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:40.858302 27127 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:41.432219 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65166.5us
I20260709 12:14:41.432596 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:41.433338 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.436126 23392 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:41.436794 27036 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.437635 23313 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:41.437688 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:40.693372 27043 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:41.440361 27188 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.081s	user 0.002s	sys 0.002s
I20260709 12:14:41.440939 26782 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.435616 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.445351 23388 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:41.445802 24009 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:41.431743 27067 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.446698 27070 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.424171 27007 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.447180 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:41.434020 27165 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11.
I20260709 12:14:41.448014 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.425917 26864 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:41.453207 24007 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:40.711563 26702 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.458441 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.436352 27081 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:41.462162 23393 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.463927 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:41.457836 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:41.465592 27134 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.864s	user 0.010s	sys 0.005s
I20260709 12:14:41.472477 23388 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:41.473773 24014 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:41.473781 24015 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:14:41.468089 27144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.026s
I20260709 12:14:41.468128 27070 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:41.480770 23395 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.481647 23391 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:41.484297 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51792.6us
I20260709 12:14:41.478288 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:41.478416 26925 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.820s	user 0.008s	sys 0.004s
W20260709 12:14:41.478875 27144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.000s	sys 0.027s
I20260709 12:14:41.490329 23390 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:41.459400 27127 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.482661 27165 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:41.493710 23389 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:41.461386 27147 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.475615 27134 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.481137 26980 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:41.472054 27108 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.496083 23387 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:41.500082 24011 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 3.
I20260709 12:14:41.485580 27003 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:41.501605 23391 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:41.507615 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.508256 23389 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.509824 23313 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:41.510660 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.512217 23321 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:41.512645 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:41.507409 27186 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.144s	user 0.000s	sys 0.002s
I20260709 12:14:41.490010 27081 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.516295 23319 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:14:41.517238 27189 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.146s	user 0.002s	sys 0.000s
I20260709 12:14:41.519917 27147 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.529866 27165 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:41.532997 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:41.511672 26980 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:41.528594 27161 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.003s
I20260709 12:14:41.532384 27190 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.529846 26925 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:41.534624 27036 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:41.548271 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47207us
I20260709 12:14:41.542305 27007 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.550849 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 906) took 1038 ms. Trace:
I20260709 12:14:41.554811 23312 rpcz_store.cc:276] 0709 12:14:40.512876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.523853 (+ 10977us) service_pool.cc:224] Handling call
0709 12:14:40.524347 (+   494us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:40.553403 (+ 29056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:40.553489 (+    86us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:40.649205 (+ 95716us) log.cc:844] Serialized 26 byte log entry
0709 12:14:41.419983 (+770778us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:41.420030 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.420109 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:41.487143 (+ 67034us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:41.549088 (+ 61945us) raft_consensus.cc:1685] finished
0709 12:14:41.549107 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.550777 (+  1670us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17398,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":30298,"spinlock_wait_cycles":135465856,"thread_start_us":742584,"threads_started":1,"wal-append.queue_time_us":694681}
I20260709 12:14:41.543538 27043 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.547178 27161 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.573220 23395 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:41.574474 26782 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.574819 27067 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.575932 26702 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:41.576386 26782 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:41.577591 26980 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.578078 27108 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.578752 27161 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.580963 27161 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.541720 27165 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.576984 26782 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.543525 27190 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.569783 27036 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:41.551915 26925 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.598670 23389 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:14:41.610505 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 173357us
I20260709 12:14:41.611661 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:41.613029 26864 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:41.597685 27134 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:41.617763 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.621767 27127 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.624581 27036 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:41.670396 27081 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:41.673846 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.674504 26782 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:41.682116 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:41.682741 26702 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.683542 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:41.684494 27007 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:41.684672 27199 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:41.686146 27165 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.687659 27190 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.653875 27161 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:14:41.691452 27161 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:41.691815 27161 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:41.681838 26864 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:41.685070 27007 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.685209 27199 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.693509 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:41.700634 23393 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:41.692703 27161 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.714650 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:41.718355 23387 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:41.709002 27003 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:41.693118 27036 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.722889 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:41.706063 27007 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:41.683555 26702 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:41.724733 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.708578 27199 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.720881 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:41.734133 23320 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:41.736994 23312 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 4.
I20260709 12:14:41.742549 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:41.708403 27210 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:41.720098 27195 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
W20260709 12:14:41.784394 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 100889us
I20260709 12:14:41.788393 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.795100 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.795913 23462 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:41.742620 27193 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:41.772684 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.000s (SENT) [suppressed 13 similar messages]
I20260709 12:14:41.802875 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.771338 27190 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.798918 27197 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:41.810351 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44906.5us
I20260709 12:14:41.811123 23393 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:41.813781 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:41.814204 23393 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:41.814853 23389 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:41.817548 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.818126 23394 raft_consensus.cc:2410] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:14:41.816045 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.818964 23465 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:41.823307 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 106999us
I20260709 12:14:41.816826 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.828177 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.832613 23470 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 5. Current term is 6. Ops: []
W20260709 12:14:41.832383 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 216364us
I20260709 12:14:41.846843 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.849267 27165 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:41.849785 27165 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:41.850807 27165 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.851305 27202 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.852150 27165 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:41.851980 27202 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.857332 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1729) took 1132 ms. Trace:
I20260709 12:14:41.857743 23394 rpcz_store.cc:276] 0709 12:14:40.724518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.725787 (+  1269us) service_pool.cc:224] Handling call
0709 12:14:41.857109 (+1131322us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:41.857252 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004673920}
I20260709 12:14:41.859155 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.859165 27202 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:41.860059 23467 raft_consensus.cc:2368] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:41.860553 23465 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:41.774289 27210 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.867455 27210 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.836261 27190 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.870609 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.871275 23465 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:41.871394 27190 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.815565 26925 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:41.832726 27161 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:41.880934 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.882534 23389 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:41.839049 26702 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.887784 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.802846 27003 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:41.891834 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 238699us
I20260709 12:14:41.894424 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:41.895833 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.897794 23470 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:41.899436 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 85484.8us
I20260709 12:14:41.884023 23393 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:41.902417 23466 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:41.905078 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:41.906738 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:41.889858 27070 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:14:41.911568 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:41.917943 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:41.925642 23465 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:41.928895 23468 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:41.929353 23393 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:41.929629 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:41.942948 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:41.919121 26702 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:41.947382 24016 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:14:41.944851 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.951475 23392 raft_consensus.cc:2368] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:41.956483 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:41.957235 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.957659 23390 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:41.933804 27203 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:14:41.959291 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:41.960117 24015 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:41.911607 27190 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:14:41.953843 27211 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:14:41.967038 23466 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:41.970247 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.952759 26702 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.954012 27127 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:41.937551 27219 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:41.976094 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:41.980795 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62257.3us
I20260709 12:14:41.960312 27203 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.953828 27194 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:41.984030 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:41.983071 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89449.3us
I20260709 12:14:41.982821 23389 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:14:41.991070 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.991604 27199 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:41.970259 27036 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:41.984477 27203 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:41.995553 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:41.996217 23392 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:41.996519 23392 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:42.002007 23393 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:41.978698 26702 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.003131 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.004232 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.004681 23391 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:42.053820 23390 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:42.015421 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:42.022107 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.022707 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:42.022975 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.025857 27199 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.030828 27081 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:42.032837 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.033537 27219 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.006273 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.006520 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.022732 23392 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.111940 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:42.113147 23389 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:42.114102 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 81374.8us
I20260709 12:14:42.061759 26864 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:14:42.119238 23392 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:42.055872 27186 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.070s	user 0.002s	sys 0.001s
W20260709 12:14:42.070158 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:42.061196 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 97096.2us
I20260709 12:14:42.061081 23388 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:42.104116 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118173us
I20260709 12:14:42.035068 27224 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:42.061295 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 96500.4us
W20260709 12:14:42.106143 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48308.3us
I20260709 12:14:42.107601 24010 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:14:42.125571 23470 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:42.126955 23287 leader_election.cc:400] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:42.127256 23287 leader_election.cc:403] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:14:42.061134 27036 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:42.129256 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.118286 27023 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:42.142588 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 119964us
I20260709 12:14:42.121925 27165 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:42.126598 27194 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.152679 27134 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:42.157622 27165 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:42.159205 27219 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.161517 27161 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.162139 27161 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.164300 27161 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.169018 27223 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.136287 27190 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:42.183420 23389 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:42.186259 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.186393 24015 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
W20260709 12:14:42.170636 27228 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:14:42.158850 27165 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.192020 27199 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:14:42.192377 27198 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:42.202772 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55526.4us
I20260709 12:14:42.194324 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.035370 27193 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 12:14:42.126731 27081 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.150964 27036 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:42.209123 23315 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:42.214501 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.215730 27081 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.215849 27003 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:42.202628 27199 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.138727 27224 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.217026 27199 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.220000 27198 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.223222 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.169639 27223 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.213457 27144 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:42.227061 24012 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.222244 26925 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:42.171414 27194 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:42.252969 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:42.250890 27241 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.066s	user 0.000s	sys 0.009s
I20260709 12:14:42.256673 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.259781 23464 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.255400 27003 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.271219 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.252691 27201 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.212774 27190 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.283629 27036 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.284063 27003 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.284710 27244 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.286038 27217 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:42.286181 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.287626 26702 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.289914 27243 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:14:42.290717 27007 consensus_meta.cc:287] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.001s
I20260709 12:14:42.291051 27201 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.293352 27197 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:42.309913 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.251634 27081 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:14:42.311357 27199 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:42.311586 23464 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:14:42.315961 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 173983us
I20260709 12:14:42.330873 24010 raft_consensus.cc:2410] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:14:42.331657 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.327504 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.311102 23392 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:42.361344 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:42.362571 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:42.260443 27203 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:42.368582 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.369467 24016 raft_consensus.cc:2368] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:42.375008 23395 raft_consensus.cc:2410] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 12:14:42.322917 27231 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:14:42.376260 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:42.377566 23982 leader_election.cc:336] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.548s (SENT)
I20260709 12:14:42.396198 23462 raft_consensus.cc:2410] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:14:42.395991 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:42.379881 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.381124 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.381394 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.396987 23470 raft_consensus.cc:2368] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:42.382474 27134 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:42.329110 27244 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.383083 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.382474 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.384014 27223 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.316632 27144 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.326169 26702 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.390861 27201 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:42.381136 27194 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:42.393307 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.398214 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.397063 24014 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:42.419086 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:42.408710 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 91566.2us
I20260709 12:14:42.409755 23388 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:42.411401 23465 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:42.315085 27217 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.415356 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.361207 27007 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.416975 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.419838 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.522878 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.523882 23318 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:42.472347 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.461304 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.528970 23316 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.468971 23321 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:42.410823 27209 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:42.509469 27144 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.458509 27201 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:42.533524 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.534726 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.535405 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.536033 23468 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
W20260709 12:14:42.540613 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 172845us
I20260709 12:14:42.472690 23462 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:42.422277 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.546502 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.548233 23388 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:42.494725 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:42.549757 23982 leader_election.cc:336] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.585s (SENT)
I20260709 12:14:42.394722 27224 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:42.559127 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.563150 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.563998 23321 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:14:42.564617 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:42.565274 23316 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:42.478346 27134 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.566395 27134 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.567613 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.568341 24013 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:42.568602 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.568882 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.569375 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.569445 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.570204 23392 raft_consensus.cc:2368] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:14:42.570726 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.571211 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.571545 24014 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:14:42.559685 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.570096 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.573699 24009 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:42.576040 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.576694 23392 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:42.536556 27198 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.523042 27244 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.560034 23317 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:42.577805 27217 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.578963 26702 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:42.579959 23387 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:42.580610 24011 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:42.510622 27203 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.550272 27209 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.566681 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.568925 27007 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.583827 24012 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:42.585652 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.568604 27134 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.588120 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.550289 27201 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:42.590747 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:42.553977 27190 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.593132 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.599852 23388 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:42.601207 23395 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:42.601292 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.586643 27248 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:42.601112 27224 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:42.606688 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:42.609676 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.610569 23387 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:14:42.609254 27224 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.614272 27224 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.614904 27193 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:42.615017 26925 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:42.615379 27193 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.615521 26925 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:42.616324 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 345161us
I20260709 12:14:42.616596 26925 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.618116 27210 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:42.618528 27210 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:42.618906 26782 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:42.619226 27194 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:42.619380 26782 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:42.619931 26782 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.619410 27210 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.620862 27210 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.621042 27198 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.614027 27023 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:42.621663 27254 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:42.622094 27254 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.621657 27198 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.622933 27254 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.623775 27161 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:42.624311 27161 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.624562 27198 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.625228 27161 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.627960 27197 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.628473 27197 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.629722 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:42.630716 27197 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.602435 27134 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:42.631722 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:42.633605 27253 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:42.634171 27253 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.634399 27193 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.635097 27253 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.635828 27193 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:42.636616 27067 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:42.637315 27224 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.638813 27197 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.644311 27197 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.602216 27201 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.648370 27201 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.640146 23320 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:42.640548 27248 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:42.648934 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.649755 27248 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.652451 27197 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.631677 27134 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.604813 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.585875 27203 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:14:42.653884 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:42.664141 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59397.5us
W20260709 12:14:42.695398 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77105.3us
I20260709 12:14:42.624305 27254 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:42.693148 27235 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.441s	user 0.000s	sys 0.005s
I20260709 12:14:42.705974 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.708077 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.709455 23315 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:42.703558 27243 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:42.710493 24015 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:42.693166 27201 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:42.712647 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:42.714998 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.715399 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.715731 23387 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.716207 23468 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:42.717183 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.717372 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.718876 23314 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:42.701342 26925 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:42.719039 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.717733 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.722589 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.723214 23470 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:42.724347 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.707914 27223 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:42.728962 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.726177 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.729950 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.731063 23318 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:42.731794 23470 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:42.732088 23389 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:42.733779 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.734977 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.735567 23468 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:42.735711 27198 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:42.736218 27198 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:42.736594 27198 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:42.737131 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.737762 23468 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:42.743913 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.745357 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.598390 27209 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.764089 23314 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:42.766268 23468 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
W20260709 12:14:42.770802 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.775964 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.776697 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.777733 23387 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:42.778224 27202 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:42.778566 27243 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.779685 27202 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:42.780184 27202 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.777609 27023 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:42.769191 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.781687 27134 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.782277 27023 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:42.782546 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:42.783310 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.784042 27203 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:42.784253 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.783387 27007 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:42.785084 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:42.785647 27260 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:42.786182 27260 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.786294 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:42.728825 27223 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.787197 27260 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.788635 27147 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:42.788530 27223 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.789098 27147 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.789640 27196 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:42.789816 27147 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:42.790242 27196 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.720844 27165 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:42.782675 27023 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.784539 27203 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.791766 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.786373 27217 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:42.797160 27196 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.781020 27202 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.725370 27201 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.785216 27007 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:42.806149 27007 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.809706 27203 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:42.810336 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.810153 27201 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.810954 23387 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:42.811657 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:42.812284 23395 raft_consensus.cc:2376] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12. Re-sending same reply.
I20260709 12:14:42.813206 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.813397 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.813973 23394 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:42.814857 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:42.795447 27165 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.817833 27165 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:42.809739 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.801262 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:42.825801 23321 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:42.839069 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.840529 23388 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:14:42.842029 27193 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:14:42.842248 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.842638 27193 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:42.853683 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:42.855468 23317 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:14:42.805620 27200 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:42.805506 26702 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:42.860091 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:42.844179 27081 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:42.852767 27244 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.854013 27224 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:42.857321 27144 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.871726 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.865756 27200 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:42.885604 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:42.800450 27023 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.865311 26782 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.888696 27224 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.903962 27224 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.905122 26782 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:42.908821 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:42.909584 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:42.897001 27165 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.909904 23389 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:42.877828 27144 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.912422 27144 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.914007 27165 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:42.914315 26702 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.914924 26702 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.917621 27161 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.918318 27161 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.898203 27261 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:42.898707 27200 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.925524 27193 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.880626 27244 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:42.901556 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.897926 27023 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:42.902928 27147 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:42.908608 27198 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:42.938373 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46833.6us
I20260709 12:14:42.939126 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:42.941987 23392 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:42.943418 23315 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:42.906962 27224 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.958030 24012 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:42.959839 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.960022 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:42.960484 23395 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:42.964291 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.964512 23317 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:42.965521 24016 raft_consensus.cc:2393] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:42.933588 27161 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:42.968775 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:42.970743 27244 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:42.975672 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:42.962038 23436 connection.cc:256] RPC call timeout handler was delayed by -1.2465389100000266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1675s for a response
W20260709 12:14:42.977344 23436 connection.cc:256] RPC call timeout handler was delayed by -1.1790389099999175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 12:14:42.979923 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:42.982084 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:42.982743 23468 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:42.990847 27198 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:42.955662 27165 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:14:42.994233 27165 consensus_queue.cc:1243] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:14:42.967361 27210 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:42.968226 26702 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:43.004877 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.005649 23394 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:43.009117 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.009923 23362 leader_election.cc:400] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:43.009967 23314 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:42.993779 24011 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:43.026178 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.026880 23313 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.026979 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.028017 23316 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:43.010519 23362 leader_election.cc:403] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:14:43.035398 23468 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:43.036793 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:43.042910 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.043182 26702 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:43.043660 23318 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:43.043780 26702 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:43.044318 23313 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:42.973482 27147 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.047923 27223 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:42.987844 27203 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.056241 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:43.056926 23436 connection.cc:256] RPC call timeout handler was delayed by -1.0233665270000074s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 12:14:43.044490 27210 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:43.055559 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.067296 23319 pending_rounds.cc:85] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
W20260709 12:14:43.061396 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 91882.7us
W20260709 12:14:43.048103 27255 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:14:43.069895 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.089814 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.094231 23216 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:43.096177 23216 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 5, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 5 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:43.104808 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.101092 23468 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:43.105912 23468 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:43.109335 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.110121 23314 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:43.112380 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:43.113579 27023 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:43.114610 27196 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:43.114969 27196 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:43.107250 23468 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.115715 23468 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:43.115794 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.115871 27196 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.119235 27196 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:43.122584 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.113811 27272 raft_consensus.cc:2764] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Leader election decision vote started in defunct term 12: won
I20260709 12:14:43.124670 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:43.071060 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.067721 23319 pending_rounds.cc:107] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:14:43.125844 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.126245 23319 raft_consensus.cc:2889] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:43.070603 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.127152 23318 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.062942 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.127521 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:43.128414 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:43.130759 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60783.9us
I20260709 12:14:43.131700 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:43.128679 23313 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:43.136181 23468 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 12:14:43.137603 23468 pending_rounds.cc:85] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 7
I20260709 12:14:43.138288 23468 pending_rounds.cc:107] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:14:43.138996 23468 raft_consensus.cc:2889] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:43.142175 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.129894 23216 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:43.147311 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:43.068961 26925 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:43.147965 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.148543 23317 raft_consensus.cc:2376] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7. Re-sending same reply.
I20260709 12:14:43.045337 27007 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.153805 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.154359 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:43.127914 27199 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:43.160840 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.139904 27165 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:43.165855 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.140192 27272 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:43.166000 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.166030 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:43.168926 23216 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:43.168010 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.170082 23387 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:43.172847 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.047999 26702 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.173353 23312 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 5: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:43.070309 27210 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.174669 23387 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:43.182029 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:43.255851 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.255630 23470 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:43.256420 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74510us
I20260709 12:14:43.256596 23313 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:43.257172 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:43.175544 23216 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:43.177148 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:43.241219 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.175866 24015 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:43.175814 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.261003 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:43.176754 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.261935 23389 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:43.149201 27197 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:43.264768 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80692.5us
I20260709 12:14:43.264950 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.267982 23391 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:14:43.161314 27081 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:43.272591 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.265513 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:43.275965 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.276162 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.280687 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:43.145646 27223 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 12:14:43.285552 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 130264us
I20260709 12:14:43.289618 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.166333 27165 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:43.309193 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.345978 23392 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:43.347333 27197 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:43.173713 27199 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.348291 27259 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.348686 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.257761 27023 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:43.350447 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.370198 23464 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:43.264243 27161 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:43.372283 23465 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:43.372514 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.376528 27203 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.377243 27203 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:43.377626 27203 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.377986 27203 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.383620 27259 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.384567 27197 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:43.385980 27259 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.386364 27007 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.389708 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.390571 23466 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:43.362913 27165 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:43.399267 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.400009 23393 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:43.277803 26925 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:43.356388 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.401314 23462 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:43.367087 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:43.367910 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.371040 23469 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:43.371321 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:43.392880 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44336us
I20260709 12:14:43.403604 23287 leader_election.cc:400] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7.
I20260709 12:14:43.175298 27036 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:43.394791 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.398345 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.392755 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:43.406883 27081 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:43.409736 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.524s (SENT)
I20260709 12:14:43.410331 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:43.419456 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.420459 23461 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:43.374816 27209 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:43.421447 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:43.392587 27197 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:43.431156 23216 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 4, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 4 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:43.440060 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.441043 24012 raft_consensus.cc:2410] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:43.423250 27260 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:43.448295 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:43.448949 23393 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:43.451370 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.428344 27023 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:43.451973 23470 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:43.388566 27161 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.427487 27197 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.426071 27081 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.443090 27198 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:43.465781 24016 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:43.465765 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:43.472050 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 214934us
I20260709 12:14:43.472277 24012 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:43.448717 27260 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:43.476876 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 211461us
I20260709 12:14:43.476894 23463 raft_consensus.cc:2368] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6.
I20260709 12:14:43.482815 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.463754 27281 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.486397 24009 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:43.486975 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 76295.2us
I20260709 12:14:43.483791 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:43.453949 27272 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:14:43.497817 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.485015 27260 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:43.499846 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:43.504858 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:43.505462 23317 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:43.507798 24012 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:43.503897 23470 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:43.509764 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:43.511555 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.513875 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:43.514637 24008 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.516450 23391 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:14:43.516531 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:43.518091 23391 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:43.518385 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:43.521203 23391 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:14:43.517653 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:43.508081 27210 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.536695 23318 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:43.518028 27254 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:14:43.493374 27281 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.513968 27194 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:14:43.553148 24008 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:43.441643 27261 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:14:43.579101 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.545594 27288 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:43.539251 27254 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:43.583607 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 225151us
I20260709 12:14:43.540189 27247 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:43.587484 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.588289 23318 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:43.554888 27210 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:43.589798 23470 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:14:43.590420 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 72861.4us
I20260709 12:14:43.579492 27200 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.593600 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.593101 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:43.595789 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:43.596374 23463 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:43.595201 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:43.589876 27254 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:43.589360 27217 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.585163 27244 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:43.591620 27193 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:43.598445 27286 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:14:43.593763 27194 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:43.598903 27253 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:43.589679 27288 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.605580 27217 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.620652 27217 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:43.610419 27283 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:14:43.599928 27282 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:43.608947 27197 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:43.613767 27194 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.577452 27287 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:14:43.636438 27224 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:43.689977 27224 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:43.690706 27224 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.651539 23394 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:43.606232 27081 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.594390 27210 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.605926 27193 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.670877 27282 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.687978 27281 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:43.598234 27200 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.690902 27287 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:14:43.702230 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107087us
I20260709 12:14:43.674857 27217 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.705076 23315 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:14:43.600824 27247 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:43.743603 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.691783 27194 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:43.762681 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:43.641233 27260 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.696997 27193 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:43.762092 27196 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.000s
I20260709 12:14:43.699704 27282 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:43.763451 23982 leader_election.cc:400] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:14:43.784152 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:43.777261 23319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.005s	sys 0.009s
W20260709 12:14:43.785071 23319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.005s	sys 0.009s
W20260709 12:14:43.785974 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 324083us
W20260709 12:14:43.787135 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 190406us
W20260709 12:14:43.867971 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 91466.3us
I20260709 12:14:43.869287 23362 leader_election.cc:400] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:43.777112 27281 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.895340 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107389us
W20260709 12:14:43.895406 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 106239us
I20260709 12:14:43.760409 27217 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.905298 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:43.788120 27197 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.786381 27199 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.784812 27081 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.909024 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 398224us
I20260709 12:14:43.911160 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.889060 26925 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:43.917032 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.917273 27197 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.926527 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:43.927601 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.910126 27281 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.926728 27007 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:43.936661 23317 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:43.928416 27197 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.942440 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:43.954758 27200 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:43.955611 26925 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.957983 27311 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.958508 27311 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:43.951418 27197 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.960583 27209 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.002s
I20260709 12:14:43.960661 27311 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:43.960072 27197 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:43.953931 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 169886us
W20260709 12:14:43.975232 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:43.908470 27196 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:14:43.961130 27209 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:43.915124 27303 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:14:43.920149 27081 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.945340 27310 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:44.011861 27194 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.014662 27081 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:44.014914 23465 raft_consensus.cc:2368] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11.
I20260709 12:14:43.962837 27197 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:44.007912 27203 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 12:14:43.961422 27311 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:43.987247 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:43.957231 27254 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:43.961936 27210 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:44.046674 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 443340us
I20260709 12:14:44.054273 23394 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:14:44.057344 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 70169.4us
I20260709 12:14:44.063448 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:44.041761 27200 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.068022 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:44.015663 27304 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:44.073163 23387 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:14:44.045157 27261 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:44.110937 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:44.119042 23387 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:14:44.158058 23463 consensus_meta.cc:287] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.000s
I20260709 12:14:44.173117 23463 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:44.067018 27303 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:44.053063 27203 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:44.219116 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 291457us
I20260709 12:14:44.815100 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:44.464244 27165 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.032s	user 0.012s	sys 0.001s
I20260709 12:14:44.781404 27302 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:44.303447 27278 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.850s	user 0.002s	sys 0.000s
W20260709 12:14:44.309175 27295 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.541s	user 0.000s	sys 0.001s
I20260709 12:14:44.198179 27247 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:44.366358 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.318114 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:44.825352 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
W20260709 12:14:44.825390 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:44.169759 27260 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:44.282253 27308 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:14:44.196725 23394 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:44.816215 27303 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.816398 27203 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:44.860175 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.775s	user 0.016s	sys 0.027s
I20260709 12:14:44.317014 27288 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:44.307907 27254 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:44.111692 27311 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:44.820187 27165 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.822136 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2422) took 1647 ms. Trace:
W20260709 12:14:44.817520 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 912275us
I20260709 12:14:44.306704 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:44.069661 27299 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:44.241235 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 691) took 1049 ms. Trace:
I20260709 12:14:44.828840 27198 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:44.332314 27280 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.846s	user 0.005s	sys 0.015s
I20260709 12:14:44.832237 27036 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.833720 27296 consensus_queue.cc:1059] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:44.834818 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:44.937585 23395 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:44.303568 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:44.860668 23433 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.023s	sys 0.034s
W20260709 12:14:44.938504 23433 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.024s	sys 0.034s
I20260709 12:14:44.820169 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2225) took 1811 ms. Trace:
W20260709 12:14:44.861126 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.016s	sys 0.027s
I20260709 12:14:44.121847 27261 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:44.866061 23391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.004s	sys 0.014s
W20260709 12:14:44.940496 23391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.005s	sys 0.014s
W20260709 12:14:44.315256 27224 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.005s
I20260709 12:14:44.323606 23461 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:14:44.940284 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 855436us
I20260709 12:14:44.872509 23396 rpcz_store.cc:276] 0709 12:14:43.174450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.174627 (+   177us) service_pool.cc:224] Handling call
0709 12:14:43.274487 (+ 99860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.274673 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:43.794609 (+519936us) log.cc:844] Serialized 14 byte log entry
0709 12:14:44.821009 (+1026400us) log.cc:844] Serialized 14 byte log entry
0709 12:14:44.821353 (+   344us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:44.821471 (+   118us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:44.821556 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.821735 (+   179us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.821999 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67459,"prepare.queue_time_us":301333,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":97,"raft.queue_time_us":690294,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":607,"thread_start_us":625735,"threads_started":2}
I20260709 12:14:44.872046 27288 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.288789 27217 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:44.104219 27200 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.312218 27194 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:44.705678 27318 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:44.868763 27244 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.014s	sys 0.033s
I20260709 12:14:44.881219 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:44.883234 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68269.3us
I20260709 12:14:44.876974 24016 rpcz_store.cc:276] 0709 12:14:43.174069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.174240 (+   171us) service_pool.cc:224] Handling call
0709 12:14:44.223132 (+1048892us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54006eebd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:44.223295 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442111744}
I20260709 12:14:44.863628 27303 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:44.863252 27304 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:44.302465 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:44.430670 27313 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.932726 27198 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:44.896832 27193 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.008s	user 0.009s	sys 0.005s
I20260709 12:14:44.876879 27299 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.939061 23319 rpcz_store.cc:276] 0709 12:14:43.008236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.008341 (+   105us) service_pool.cc:224] Handling call
0709 12:14:43.056970 (+ 48629us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:43.129915 (+ 72945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:43.129948 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:43.143575 (+ 13627us) log.cc:844] Serialized 78 byte log entry
0709 12:14:43.785544 (+641969us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:43.785600 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:43.785672 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:44.374293 (+588621us) raft_consensus.cc:1685] finished
0709 12:14:44.374326 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:44.820075 (+445749us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4388,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":12123,"thread_start_us":612108,"threads_started":1,"wal-append.queue_time_us":940301}
I20260709 12:14:44.817907 27302 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:44.951548 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 116828us
I20260709 12:14:44.952500 27224 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:44.941200 27282 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.072s	user 0.000s	sys 0.012s
I20260709 12:14:44.984679 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:44.952318 27281 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:44.879021 27311 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:44.980134 27244 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.018s	sys 0.036s
I20260709 12:14:44.944197 27036 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.953704 27288 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.954087 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 767) took 1361 ms. Trace:
I20260709 12:14:45.063081 27282 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:45.062686 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 78115.2us
I20260709 12:14:44.965288 27247 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:44.972214 27326 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.972283 27320 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.061932 23461 rpcz_store.cc:276] 0709 12:14:43.588060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.588197 (+   137us) service_pool.cc:224] Handling call
0709 12:14:44.290026 (+701829us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:44.290309 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.944471 (+654162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.949379 (+  4908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5657984}
I20260709 12:14:44.977844 27254 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:45.096113 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.100613 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1764) took 1064 ms. Trace:
I20260709 12:14:44.996554 27299 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:44.977579 27318 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.029321 27296 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:45.017791 27194 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:45.062178 27319 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.021360 27327 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.026322 27260 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:45.105783 27323 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:14:45.143580 23388 rpcz_store.cc:276] 0709 12:14:44.036133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.295265 (+259132us) service_pool.cc:224] Handling call
0709 12:14:44.295663 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:44.828293 (+532630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:44.828328 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:44.828906 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:14:44.992884 (+163978us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:44.992938 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.993016 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.100225 (+107209us) raft_consensus.cc:1685] finished
0709 12:14:45.100357 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.100537 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530925,"raft.queue_time_us":536006,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":163599,"threads_started":1,"wal-append.queue_time_us":234122}
W20260709 12:14:45.146536 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 265409us
I20260709 12:14:45.160353 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:45.149863 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53854.3us
I20260709 12:14:45.163318 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.017547 27313 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:44.996438 27193 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.141377 27196 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:14:45.142840 27244 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:14:45.148805 27254 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.206787 23313 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:14:45.212711 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:45.213536 23313 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:45.223158 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:45.165782 27260 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:45.234884 27329 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:14:45.153414 27259 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:14:45.235576 27200 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.159432 27327 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.236302 27260 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.236853 27259 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:45.237013 27260 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:14:45.237300 27259 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:45.156726 27194 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:45.237500 27322 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:14:45.160097 27319 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.259214 27319 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:45.238385 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 95522.1us
I20260709 12:14:45.160403 27336 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:45.266180 27336 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:45.267127 27336 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.267866 27336 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.201784 27282 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.165691 27165 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.274607 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:45.275434 23318 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 5.
I20260709 12:14:45.277292 23313 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:14:45.277732 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 291697us
I20260709 12:14:45.146095 27320 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.154562 27296 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.245481 27327 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:45.280054 27331 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:14:45.280354 27318 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:45.281684 27332 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:14:45.281896 27254 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:45.282160 27330 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 12:14:45.280364 27334 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:45.283421 27302 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.235595 27193 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.323385 24015 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:14:45.283901 27194 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.284705 27288 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.286079 27311 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.289165 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2427) took 1907 ms. Trace:
I20260709 12:14:45.290696 27313 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.326521 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.333530 24009 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:45.291708 27036 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.292512 27224 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.350534 27224 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.351015 27224 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.352062 27317 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:45.352662 27200 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:45.293037 27198 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.352960 27301 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:45.293501 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 696) took 1824 ms. Trace:
I20260709 12:14:45.295574 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:45.353856 24016 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:45.239778 27261 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.151057 27326 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.356138 27320 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.358165 27326 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.317358 27303 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.273293 27259 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.360355 27259 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:45.361333 27303 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.319144 27201 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.322228 27318 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:45.362355 27201 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.362483 27318 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.364516 27201 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.364717 27318 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.328042 23393 rpcz_store.cc:276] 0709 12:14:43.382028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.459431 (+ 77403us) service_pool.cc:224] Handling call
0709 12:14:43.470344 (+ 10913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.486782 (+ 16438us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:43.578710 (+ 91928us) log.cc:844] Serialized 14 byte log entry
0709 12:14:43.683637 (+104927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:43.683715 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:43.683797 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.288893 (+1605096us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b54002af8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:45.288941 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.289097 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":102812,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":1586,"raft.queue_time_us":2885,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"spinlock_wait_cycles":37298560,"thread_start_us":180858,"threads_started":2,"wal-append.queue_time_us":96965,"wal-append.run_cpu_time_us":1449,"wal-append.run_wall_time_us":1024078}
W20260709 12:14:45.366103 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 70604.7us
I20260709 12:14:45.324559 27244 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:45.368423 27244 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.369678 27261 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:45.370411 27261 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:45.370868 27244 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.371006 27193 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:45.346670 27338 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.325423 27321 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:45.349502 27296 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.324858 27223 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:45.366201 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2231) took 1062 ms. Trace:
I20260709 12:14:45.348834 24012 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:45.386509 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:45.353219 27224 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.384191 23315 rpcz_store.cc:276] 0709 12:14:44.304213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.305368 (+  1155us) service_pool.cc:224] Handling call
0709 12:14:44.305794 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:44.306238 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:44.306279 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:44.819433 (+513154us) log.cc:844] Serialized 26 byte log entry
0709 12:14:44.866978 (+ 47545us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:44.867047 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:44.867126 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:45.365920 (+498794us) raft_consensus.cc:1685] finished
0709 12:14:45.365949 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.366123 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2803,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"thread_start_us":47110,"threads_started":1,"wal-append.queue_time_us":132427}
I20260709 12:14:45.393889 27223 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:45.395442 27335 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:14:45.395103 27223 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.352653 27317 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:45.376456 23436 leader_election.cc:336] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.501s (SENT)
I20260709 12:14:45.371574 27261 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:45.381769 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:45.400394 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:45.382313 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 1.31426e+06us
I20260709 12:14:45.401389 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.401885 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.406001 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:45.388314 27244 raft_consensus.cc:2764] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:14:45.412202 27007 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:45.353250 27200 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.416651 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1758) took 1486 ms. Trace:
I20260709 12:14:45.383510 27338 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.417587 27253 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:45.418027 27253 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:45.418143 27304 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.418486 27253 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:45.418749 27304 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.417052 23391 rpcz_store.cc:276] 0709 12:14:43.930170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.934410 (+  4240us) service_pool.cc:224] Handling call
0709 12:14:43.939627 (+  5217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.939928 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:43.972381 (+ 32453us) log.cc:844] Serialized 14 byte log entry
0709 12:14:44.994262 (+1021881us) log.cc:844] Serialized 14 byte log entry
0709 12:14:45.416181 (+421919us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b1800470bc0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287
0709 12:14:45.416283 (+   102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:45.416348 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:45.416400 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.416465 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.416591 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38361,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":9894,"raft.queue_time_us":188222,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4879,"spinlock_wait_cycles":201124608,"thread_start_us":1112702,"threads_started":3,"wal-append.queue_time_us":950206}
W20260709 12:14:45.444555 23284 connection.cc:256] RPC call timeout handler was delayed by -1.5928141000001688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:14:45.449843 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 73434.4us
I20260709 12:14:45.449826 27341 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:45.463310 27341 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.464018 27341 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.468544 27196 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:45.469336 27196 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.487232 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:45.524224 27224 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.353734 24010 rpcz_store.cc:276] 0709 12:14:43.469019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.469163 (+   144us) service_pool.cc:224] Handling call
0709 12:14:43.469547 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:43.469794 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:43.782914 (+313120us) log.cc:844] Serialized 14 byte log entry
0709 12:14:44.072113 (+289199us) log.cc:844] Serialized 14 byte log entry
0709 12:14:45.288127 (+1216014us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b180041de80. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:45.292939 (+  4812us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b180041de80. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:14:45.293110 (+   171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:45.293164 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:45.293219 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.293285 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.293406 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1163720,"prepare.queue_time_us":72344,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":116,"raft.queue_time_us":1315983,"raft.run_cpu_time_us":446,"raft.run_wall_time_us":86238,"spinlock_wait_cycles":92816512,"thread_start_us":562553,"threads_started":2,"wal-append.queue_time_us":494765,"wal-append.run_cpu_time_us":3647,"wal-append.run_wall_time_us":1020788}
W20260709 12:14:45.579725 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 177939us
I20260709 12:14:45.530090 23216 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 8, leader changed from <none> to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 8 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.580910 27243 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:45.582314 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:45.581964 23216 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:45.583137 27243 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:14:45.583264 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 181963us
I20260709 12:14:45.583628 27243 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.584080 27243 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:45.584148 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.584363 27336 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.584980 27203 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:45.585462 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:45.586119 27317 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.586292 27203 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:45.586707 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:45.589859 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.590181 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:45.591214 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 430909us
I20260709 12:14:45.581182 27338 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:45.583451 23214 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 316) took 24625 ms (client timeout 15000 ms). Trace:
I20260709 12:14:45.607836 23216 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 10, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:45.628763 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.388314 27209 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:14:45.622327 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1766) took 1564 ms. Trace:
I20260709 12:14:45.601127 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:45.622658 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 459442us
I20260709 12:14:45.632454 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.619354 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:45.630427 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.525058 27224 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.638882 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.639894 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:45.602190 27134 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:45.640435 23465 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:14:45.639292 23982 connection.cc:256] RPC call timeout handler was delayed by -1.2271486080001068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 12:14:45.602027 27203 raft_consensus.cc:2764] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:14:45.641129 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:45.633845 23216 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 6, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 6 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.581964 27200 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.638911 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:45.620848 27247 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:45.589010 27299 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.649854 24009 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:45.649859 27287 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:45.659766 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:45.663110 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:45.663460 24013 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:45.663730 24014 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:45.663897 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:45.665195 23982 leader_election.cc:400] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11.
I20260709 12:14:45.649562 27134 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:45.653838 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.645701 27336 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:45.652019 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:45.705255 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53322.3us
I20260709 12:14:45.654410 27318 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:45.708884 23388 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:45.710028 27342 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:45.710582 27342 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:45.711607 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:45.711463 27342 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.654647 27296 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:45.630136 23390 rpcz_store.cc:276] 0709 12:14:44.058737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.296927 (+238190us) service_pool.cc:224] Handling call
0709 12:14:44.297255 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.297502 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:14:44.812991 (+515489us) log.cc:844] Serialized 14 byte log entry
0709 12:14:45.520356 (+707365us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b180032c260. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:14:45.621822 (+101466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:45.621898 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:45.621969 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:45.622068 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:45.622266 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":514879,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":314,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":55876352,"thread_start_us":101138,"threads_started":1,"wal-append.queue_time_us":807867}
I20260709 12:14:45.628475 27161 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.658962 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:45.632491 27209 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:45.659606 24016 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:14:45.660068 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:45.725415 23465 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:45.658385 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.662201 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:45.646719 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:45.726612 24010 raft_consensus.cc:2376] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10. Re-sending same reply.
I20260709 12:14:45.704119 27134 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:45.731583 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.732844 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.734081 27345 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:45.734879 27345 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.737808 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.667719 27203 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:45.667862 27224 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.741809 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.668527 27344 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.742563 23464 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:45.742810 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.742707 27344 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:45.669070 27196 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.670002 27304 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:45.596211 23214 rpcz_store.cc:269] 0709 12:14:20.976342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.998354 (+ 22012us) service_pool.cc:224] Handling call
0709 12:14:45.583224 (+24584870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:41.219773 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:41.220667 (+   894us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:41.220735 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:14:41.220779 (+    44us) tablet.cc:662] Decoding operations
0709 12:14:41.222146 (+  1367us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:41.222257 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:41.222297 (+    40us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:14:41.223450 (+  1153us) tablet.cc:701] Row locks acquired
0709 12:14:41.223505 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:14:41.223747 (+   242us) write_op.cc:270] Start()
0709 12:14:41.223957 (+   210us) write_op.cc:276] Timestamp: P: 1783599281223656 usec, L: 0
0709 12:14:41.224012 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 12:14:41.224943 (+   931us) log.cc:844] Serialized 3654 byte log entry
0709 12:14:41.844934 (+619991us) op_driver.cc:464] REPLICATION: finished
0709 12:14:42.298126 (+453192us) write_op.cc:301] APPLY: starting
0709 12:14:42.314153 (+ 16027us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:42.384446 (+ 70293us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:42.384586 (+   140us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:42.385988 (+  1402us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:42.386294 (+   306us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 12:14:42.386339 (+    45us) write_op.cc:312] APPLY: finished
0709 12:14:42.386739 (+   400us) log.cc:844] Serialized 71 byte log entry
0709 12:14:42.386938 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:42.387066 (+   128us) write_op.cc:454] Released schema lock
0709 12:14:42.387127 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:42.387696 (+   569us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:42.387739 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":153188,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":365852,"apply.run_cpu_time_us":3912,"apply.run_wall_time_us":117763,"num_ops":9,"prepare.queue_time_us":654709,"prepare.run_cpu_time_us":11581,"prepare.run_wall_time_us":424906,"raft.queue_time_us":73145,"raft.run_cpu_time_us":34815,"raft.run_wall_time_us":322589,"replication_time_us":588404,"spinlock_wait_cycles":1408,"thread_start_us":686257,"threads_started":4,"wal-append.queue_time_us":481456,"wal-append.run_cpu_time_us":19819,"wal-append.run_wall_time_us":1683514}]]}
I20260709 12:14:45.727239 27198 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:45.678016 24013 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:45.681890 27341 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:45.682961 27247 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.744860 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.745232 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.745729 23469 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:45.746382 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.746855 23396 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:14:45.686412 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54079.7us
I20260709 12:14:45.694361 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:45.698800 23982 leader_election.cc:403] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:14:45.649776 27202 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:14:45.702286 23470 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:45.701930 23216 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:45.644932 27223 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:45.703097 24011 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:14:45.658188 27338 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:45.713979 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.773082 23391 raft_consensus.cc:2410] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:45.657914 27272 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:45.723109 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:45.724169 23392 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:45.667058 27317 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:45.728099 24007 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 4: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:45.726365 23461 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:45.742570 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.746914 24013 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.777046 24008 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
W20260709 12:14:45.701411 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47658.4us
W20260709 12:14:45.748775 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83725.6us
I20260709 12:14:45.748838 23393 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:45.763863 27340 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:14:45.765116 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:45.766319 27299 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:45.798033 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:45.799117 23388 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:45.768939 27209 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.804674 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:45.805588 23391 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:45.741854 27203 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:45.767304 27161 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11.
I20260709 12:14:45.771092 24010 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:45.777235 24013 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 10.
I20260709 12:14:45.800734 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:45.816972 23387 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:45.819355 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:45.820147 23313 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:45.811194 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.821233 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:45.821789 23392 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:45.769645 23216 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:45.814867 23462 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:45.822324 23317 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:45.816449 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.822674 23317 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:45.815145 27336 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.825825 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.832544 24013 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:45.826722 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.833578 24010 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:45.827555 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.808580 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.833873 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.834451 23389 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:45.835522 23470 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:14:45.774645 27338 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:45.843003 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.766561 27345 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:45.837908 23394 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:45.816952 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:45.824347 23393 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:14:45.847898 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:45.848062 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.849206 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.849792 23393 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:45.851814 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:45.856526 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.857098 23466 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:45.840555 23313 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:45.840821 23387 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:45.812282 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:45.823573 23317 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.843886 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.841670 23216 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 5, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:45.846961 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:45.806543 27318 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:45.774950 27223 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.860177 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:46.092685 23317 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:46.070564 23467 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:45.852240 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:45.853220 27198 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.093298 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.810628 26925 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:45.855096 23470 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 3. Current term is 4. Ops: []
I20260709 12:14:46.096433 23391 raft_consensus.cc:2410] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:14:45.858014 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:46.013726 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:45.852779 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.851840 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.098819 23390 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:46.099179 24007 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:45.852429 23315 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:14:46.098899 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.909222 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.101555 23394 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:45.852679 23388 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:45.850646 27338 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:46.104457 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.105254 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1754) took 2167 ms. Trace:
W20260709 12:14:46.104453 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 292215us
I20260709 12:14:46.106750 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:45.980221 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.108038 24008 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:46.110702 23464 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:46.118885 23392 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:46.121886 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:46.122583 23388 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:46.122579 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:46.105651 23395 rpcz_store.cc:276] 0709 12:14:43.920043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.922736 (+  2693us) service_pool.cc:224] Handling call
0709 12:14:43.931411 (+  8675us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:44.938083 (+1006672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.086668 (+1148585us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001e3f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:46.086913 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2313497344}
I20260709 12:14:45.831960 27210 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:46.128718 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.129971 23387 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:45.843381 27209 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:14:46.131960 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.130995 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.137116 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.136839 23216 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:46.137712 23391 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:46.138221 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.138588 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.851930 27343 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.139305 23470 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:46.139778 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.147102 23387 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:46.048440 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:46.148381 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:45.861711 27217 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:46.158690 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.182320 23387 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:46.159288 23392 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:46.167152 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:45.856060 27320 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:46.162518 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.185047 23391 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:46.185647 23391 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9.
I20260709 12:14:46.141567 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.186738 24011 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:46.178825 23388 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:45.849294 27341 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.096359 27201 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.191568 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.308772 23468 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:46.096186 27287 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:46.047261 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 193644us
I20260709 12:14:46.304687 24016 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:46.250896 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.307258 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.311295 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.307847 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.312378 23321 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:45.835757 27348 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.316s
I20260709 12:14:46.095043 27198 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.279037 23317 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:14:46.335584 23317 pending_rounds.cc:85] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:14:46.335985 23317 pending_rounds.cc:107] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:14:46.336364 23317 raft_consensus.cc:2889] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:46.153990 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:46.338042 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 184125us
I20260709 12:14:45.855702 27299 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:45.846884 27336 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:45.981405 27254 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.334s
I20260709 12:14:46.139962 27200 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:46.046295 27294 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:46.203648 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.319566 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.310118 27244 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:46.321230 23319 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:14:46.321830 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 476276us
I20260709 12:14:46.392457 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:46.392643 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:46.123246 27272 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.120401 27161 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:46.095765 26925 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.325724 23313 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:46.325627 23387 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:46.326387 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.328577 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.411127 23314 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:46.340507 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.415102 23468 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:46.341799 23216 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 5 to 7, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 7 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:46.347090 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.416450 23464 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:46.054138 27345 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.359198 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.366427 27200 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.373473 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:46.130645 27318 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.374147 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.349419 27341 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.176441 27210 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:46.412068 23361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.015s	sys 0.052s
W20260709 12:14:46.437242 23361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.016s	sys 0.052s
I20260709 12:14:46.376998 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:46.384886 24014 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.000s
I20260709 12:14:46.439487 24014 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:46.319643 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.440554 23462 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:46.305580 27217 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:46.395279 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.437222 23216 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 5 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:46.443581 23390 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:46.401515 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:46.403769 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.030s	sys 0.028s
I20260709 12:14:46.406055 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:46.444818 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.030s	sys 0.029s
I20260709 12:14:46.445060 23470 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:46.418674 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:46.417243 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 261673us
I20260709 12:14:46.369760 27334 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:46.357173 27299 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:46.413727 27134 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.005s
I20260709 12:14:46.455812 27134 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.456187 27217 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.457939 27134 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:46.392005 27244 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:46.417980 26925 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:46.344148 27287 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.473007 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:46.437966 27200 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:46.473613 23396 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:46.454957 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.461894 23216 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:46.341938 27201 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.253038 27343 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.355719 27320 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:46.472043 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 365341us
I20260709 12:14:46.486603 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.491046 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.451004 27334 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.474686 23470 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:46.329815 27203 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.417606 27272 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.472534 27244 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:46.500523 23216 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 7, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 7 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.508383 24007 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:14:46.488462 27299 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:46.513406 27296 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:46.512954 23216 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 7, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 7 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.537161 23216 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:46.516285 23393 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:46.517103 27272 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:46.443037 27345 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.533146 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:46.533973 23394 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:46.482151 27359 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:46.499308 27161 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:14:46.551102 27253 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.132s	user 0.031s	sys 0.019s
I20260709 12:14:46.592759 27161 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:46.593331 27161 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:46.593212 27253 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.596781 27253 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:46.533885 27358 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:14:46.550968 23216 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:46.557780 27365 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:46.562876 27354 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:14:46.577867 27357 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.427s	user 0.001s	sys 0.000s
I20260709 12:14:46.578629 23389 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:46.580324 27345 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:46.583051 27343 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:46.584133 27286 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.588734 27348 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:46.590339 27341 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:46.592105 27210 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:46.652181 24015 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.023s	sys 0.043s
W20260709 12:14:46.652673 24015 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.023s	sys 0.043s
W20260709 12:14:46.642163 27356 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.588s	user 0.001s	sys 0.000s
W20260709 12:14:46.666646 23284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.012s	sys 0.022s
I20260709 12:14:46.633180 23216 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:46.507182 27334 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:46.643112 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:46.520341 27344 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:46.672428 23216 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:46.648378 27286 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:46.511580 27259 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 12:14:46.701997 23389 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:14:46.707032 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:46.653235 27348 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:46.708935 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:46.598978 27134 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:14:46.689862 23216 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 5, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 5 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.664111 27223 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:46.712037 23216 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 5, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 5 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.742470 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:46.689169 27342 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.024s	sys 0.015s
I20260709 12:14:46.710413 27348 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:46.711099 27198 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:46.789628 27342 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.025s	sys 0.015s
W20260709 12:14:46.827317 23469 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.026s	sys 0.025s
W20260709 12:14:46.840029 23469 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.027s	sys 0.025s
I20260709 12:14:46.799932 27336 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:46.831344 27007 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.419s	user 0.021s	sys 0.000s
W20260709 12:14:46.839970 27337 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.688s	user 0.041s	sys 0.054s
W20260709 12:14:46.852895 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 493749us
I20260709 12:14:46.862461 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.863314 27198 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:46.863745 27198 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.864084 27198 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:46.864444 27201 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:46.840585 27342 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:46.861969 27337 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.710s	user 0.041s	sys 0.054s
I20260709 12:14:46.855722 27287 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:46.859634 27007 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:14:46.898840 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 476017us
I20260709 12:14:46.900619 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.900763 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:46.912750 24012 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:46.892369 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.912079 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:46.918465 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.887692 27320 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:46.928149 23462 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:46.927429 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.930629 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.931795 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.885885 27342 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:46.943488 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.944206 23464 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:46.919751 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:46.919843 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 395548us
W20260709 12:14:46.956355 23982 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.865s (SENT) [suppressed 14 similar messages]
I20260709 12:14:46.957487 24011 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:46.958415 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.922991 27343 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:46.966240 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.923352 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:46.923484 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.924252 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:46.924840 23396 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:46.934371 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.961998 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.969390 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 675) took 1166 ms. Trace:
I20260709 12:14:46.969889 23396 rpcz_store.cc:276] 0709 12:14:45.802644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.122659 (+320015us) service_pool.cc:224] Handling call
0709 12:14:46.472858 (+350199us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:14:46.969312 (+496454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45432192}
I20260709 12:14:46.908442 27336 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:46.962172 23462 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:46.971203 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:46.975308 23436 leader_election.cc:400] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9.
I20260709 12:14:46.975636 23436 leader_election.cc:403] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:46.977003 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:46.978003 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54732.9us
I20260709 12:14:46.979792 23470 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:46.980647 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:46.926964 27337 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:46.919054 27318 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:46.926101 27287 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:46.975647 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:46.991626 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:46.995486 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:46.929046 27361 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:47.004201 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.966862 27311 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:46.902629 27369 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:14:46.931896 27320 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.973519 27261 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.009109 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.009348 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:46.992735 27287 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.991617 27286 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:47.020740 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 904) took 1373 ms. Trace:
I20260709 12:14:47.020836 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:46.934377 27341 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:14:46.990839 27210 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:46.991024 27345 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:47.024499 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5536549999999352s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.244s for a response
I20260709 12:14:46.990782 27294 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:46.974131 27360 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:46.992169 27337 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:47.030144 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.022s	sys 0.030s
I20260709 12:14:47.028070 23462 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
W20260709 12:14:47.099054 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.022s	sys 0.030s
I20260709 12:14:47.014443 23463 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:47.042037 23470 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:47.067127 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 58261.1us
I20260709 12:14:47.013422 27272 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:14:47.003377 27361 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:47.017342 27311 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.017525 27254 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:47.246942 27361 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:47.251762 27370 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
I20260709 12:14:47.042303 24015 rpcz_store.cc:276] 0709 12:14:45.647514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.647641 (+   127us) service_pool.cc:224] Handling call
0709 12:14:45.648025 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:45.648512 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:45.648556 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:45.649306 (+   750us) log.cc:844] Serialized 52 byte log entry
0709 12:14:45.834332 (+185026us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:46.653089 (+818757us) log.cc:844] Serialized 14 byte log entry
0709 12:14:46.657150 (+  4061us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.020261 (+363111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.020380 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.020445 (+    65us) raft_consensus.cc:1685] finished
0709 12:14:47.020468 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.020658 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1091327,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":374014,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":14887,"thread_start_us":1136144,"threads_started":3,"wal-append.queue_time_us":720940}
I20260709 12:14:47.246551 27360 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:47.253582 27360 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.256256 27360 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:47.256839 27360 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.979444 27343 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.111897 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:47.124794 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.039240 27287 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:47.130349 23980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.007s	sys 0.002s
I20260709 12:14:47.205953 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:47.229019 27243 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.644s	user 0.008s	sys 0.011s
I20260709 12:14:47.066972 27320 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:47.117327 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108662us
I20260709 12:14:47.263012 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:47.210824 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 965) took 1150 ms. Trace:
I20260709 12:14:47.037092 27375 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.204s
I20260709 12:14:47.282830 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:47.261912 27261 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:47.227787 24007 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:47.012475 27374 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:14:47.231007 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:47.310784 24013 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:47.078172 27210 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:47.088078 27345 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.218087 27247 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.091415 27294 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.034430 27336 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:47.218743 27244 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.002s
W20260709 12:14:47.248349 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.247157 27369 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:47.253852 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:47.020273 27318 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.257624 27311 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:47.260208 27342 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:47.235575 27382 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:47.269284 23980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.013s	sys 0.007s
I20260709 12:14:47.105774 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:47.376612 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 928211us
W20260709 12:14:47.377077 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 271407us
I20260709 12:14:47.378199 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:47.379289 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:47.379655 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:47.268596 24009 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:47.342257 27294 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:47.280200 27341 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:47.386708 23394 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:47.284077 23462 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:47.221215 24016 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:47.270630 24008 rpcz_store.cc:276] 0709 12:14:46.060768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.131468 (+ 70700us) service_pool.cc:224] Handling call
0709 12:14:47.132341 (+1000873us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b540008e8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:47.210752 (+ 78411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1429248896}
I20260709 12:14:47.297605 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:47.247087 27217 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:47.305084 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2466) took 1470 ms. Trace:
W20260709 12:14:47.311823 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 223415us
I20260709 12:14:47.238768 27286 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:47.257321 27360 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.306064 27261 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.360108 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:47.363565 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:47.322697 27210 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.365694 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2470) took 1530 ms. Trace:
I20260709 12:14:47.368081 23393 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:14:47.369639 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:47.283357 27243 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.332425 27345 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:47.098558 27378 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:47.271020 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:47.384260 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:47.384697 27399 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:47.341142 27336 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:47.389806 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:47.363030 27369 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:47.410172 24007 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:47.343364 27247 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.365346 27244 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.410863 23470 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:47.484328 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:47.412410 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:47.491000 23390 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:47.429566 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:47.484658 27320 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:47.444809 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 457709us
W20260709 12:14:47.475909 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1981549999998151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28350000000000003s for a response
W20260709 12:14:47.492973 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108791us
I20260709 12:14:47.495913 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.439006 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:47.342935 27337 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:47.467765 27377 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
W20260709 12:14:47.467180 27379 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
W20260709 12:14:47.472450 27400 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:14:47.468343 27386 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:14:47.513530 23468 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:47.489161 23466 rpcz_store.cc:276] 0709 12:14:45.835348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.046018 (+210670us) service_pool.cc:224] Handling call
0709 12:14:46.190360 (+144342us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:46.190609 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.190882 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:46.633697 (+442815us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.146864 (+513167us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b180041ba20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:14:47.261870 (+115006us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.364927 (+103057us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.364979 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:47.365036 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.365103 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.365232 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10308,"prepare.queue_time_us":722991,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":102,"raft.queue_time_us":208091,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":24759,"spinlock_wait_cycles":78870144,"thread_start_us":712961,"threads_started":3,"wal-append.queue_time_us":755045}
W20260709 12:14:47.462105 27161 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.006s
I20260709 12:14:47.504287 23394 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.507531 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:47.387861 27341 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:47.375821 27318 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:47.559666 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63861.8us
W20260709 12:14:47.468925 27393 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.207s	user 0.002s	sys 0.000s
I20260709 12:14:47.497758 27007 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:47.391695 23469 rpcz_store.cc:276] 0709 12:14:45.834276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.835598 (+  1322us) service_pool.cc:224] Handling call
0709 12:14:45.980731 (+145133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:45.981007 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:46.845147 (+864140us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.291771 (+446624us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:47.304765 (+ 12994us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:47.304816 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.304892 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.305026 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":884863,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":4435,"raft.queue_time_us":231169,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":29114,"spinlock_wait_cycles":19497216,"thread_start_us":1305314,"threads_started":3,"wal-append.queue_time_us":402062}
I20260709 12:14:47.489615 27336 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:47.627424 23317 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:47.550977 27320 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.623379 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.671126 23314 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:14:47.484336 27390 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:14:47.408933 27384 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:47.407395 27342 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.632809 23320 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:47.385707 27387 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:47.392094 27203 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:47.506749 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:47.513902 27200 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.011s	user 0.011s	sys 0.000s
I20260709 12:14:47.390587 27317 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:14:47.503664 27134 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.622534 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:47.643905 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:47.695883 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52047.2us
I20260709 12:14:47.697036 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.644846 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 901) took 1998 ms. Trace:
I20260709 12:14:47.539526 27286 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.657884 27343 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:47.659842 23466 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:47.558403 27374 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.549336 27399 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:47.558192 27253 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:47.668984 23470 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:47.673043 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.625888 27337 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:47.413915 27217 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:47.389205 27361 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:47.698911 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.699726 23394 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:14:47.653990 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.657915 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.711428 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.814812 23317 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:47.732972 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1781) took 1624 ms. Trace:
I20260709 12:14:47.711752 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:47.713061 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 784) took 1215 ms. Trace:
I20260709 12:14:47.694926 27036 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:47.697705 24012 rpcz_store.cc:276] 0709 12:14:45.646642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.646784 (+   142us) service_pool.cc:224] Handling call
0709 12:14:45.659441 (+ 12657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:46.913367 (+1253926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.644578 (+731211us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54007f0154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:47.644792 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880959104}
I20260709 12:14:47.718429 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.639256 27161 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:47.812409 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:47.840117 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 128493us
I20260709 12:14:47.840533 23316 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:47.841846 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.842926 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:47.843883 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:47.413302 26702 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.219s
I20260709 12:14:47.845479 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:47.816082 23465 rpcz_store.cc:276] 0709 12:14:46.497251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.497445 (+   194us) service_pool.cc:224] Handling call
0709 12:14:46.497778 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.498104 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.498129 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.506774 (+  8645us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.959812 (+453038us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:46.959849 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.959914 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.712815 (+752901us) raft_consensus.cc:1685] finished
0709 12:14:47.712847 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.713000 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22716,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":13524,"thread_start_us":324988,"threads_started":1,"wal-append.queue_time_us":587050}
I20260709 12:14:47.818975 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.768407 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.847267 23321 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:47.683960 27387 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:47.851187 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:47.851814 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.852154 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:47.696163 27317 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:47.853479 23391 rpcz_store.cc:276] 0709 12:14:46.109049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.186261 (+ 77212us) service_pool.cc:224] Handling call
0709 12:14:46.394180 (+207919us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:46.394385 (+   205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.408282 (+ 13897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.408328 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.448057 (+ 39729us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.902558 (+454501us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:46.902601 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.902668 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.720959 (+818291us) raft_consensus.cc:1685] finished
0709 12:14:47.720979 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.732905 (+ 11926us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52754,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":9982,"spinlock_wait_cycles":3583232,"thread_start_us":412320,"threads_started":1,"wal-append.queue_time_us":556701}
I20260709 12:14:47.387249 27382 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.390916 27363 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.467s
I20260709 12:14:47.865505 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1782) took 1741 ms. Trace:
I20260709 12:14:47.867715 27411 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:47.868360 27414 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.682186 27320 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:47.869709 27334 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.163s	user 0.007s	sys 0.007s
I20260709 12:14:47.871155 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:47.693045 27384 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:47.874585 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:47.885644 27317 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:47.709893 27203 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:47.877857 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:47.938169 23391 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:47.890869 27036 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:47.882953 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:47.717770 27134 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.888289 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.889331 24014 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:47.878068 27296 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:47.945091 23317 raft_consensus.cc:2410] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:14:47.892318 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 252793us
I20260709 12:14:47.731685 27321 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:47.911818 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:47.690861 27200 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:47.693584 27342 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.871177 27378 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:47.887781 27396 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:14:47.903348 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:47.924609 24011 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:47.697678 27254 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:47.662648 27369 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:47.870777 27202 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.733217 27374 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:47.879513 23388 rpcz_store.cc:276] 0709 12:14:46.123639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.306471 (+182832us) service_pool.cc:224] Handling call
0709 12:14:46.402784 (+ 96313us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:14:46.402956 (+   172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.418339 (+ 15383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.418383 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.443853 (+ 25470us) log.cc:844] Serialized 26 byte log entry
0709 12:14:46.893323 (+449470us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:46.893380 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:46.893512 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.817935 (+924423us) raft_consensus.cc:1685] finished
0709 12:14:47.839328 (+ 21393us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:47.865400 (+ 26072us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14983,"raft.queue_time_us":69996,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":6740,"spinlock_wait_cycles":45186816,"thread_start_us":449156,"threads_started":1,"wal-append.queue_time_us":686692}
I20260709 12:14:47.893133 27411 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:47.942644 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:47.877223 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 373242us
I20260709 12:14:48.029062 23391 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:48.029337 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:47.909085 27414 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:48.043643 27388 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.497s	user 0.000s	sys 0.001s
I20260709 12:14:48.044906 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 961) took 2200 ms. Trace:
I20260709 12:14:48.045270 27414 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.045974 27414 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:48.046420 27414 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.046782 27414 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:48.047081 27405 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:14:48.047425 23391 raft_consensus.cc:2410] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:14:48.048352 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 789) took 1077 ms. Trace:
I20260709 12:14:48.048725 23461 rpcz_store.cc:276] 0709 12:14:46.971422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.971526 (+   104us) service_pool.cc:224] Handling call
0709 12:14:46.971846 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.972154 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.972184 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.000646 (+ 28462us) log.cc:844] Serialized 26 byte log entry
0709 12:14:47.387256 (+386610us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:47.387306 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.387380 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.048161 (+660781us) raft_consensus.cc:1685] finished
0709 12:14:48.048181 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.048296 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":344,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17567,"thread_start_us":168735,"threads_started":1,"wal-append.queue_time_us":725600}
W20260709 12:14:48.050575 27198 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.186s	user 0.011s	sys 0.000s
W20260709 12:14:48.050797 27407 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
W20260709 12:14:48.052094 27403 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
W20260709 12:14:48.053092 27381 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.411s	user 0.000s	sys 0.001s
I20260709 12:14:48.017670 27409 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:48.056324 27409 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:48.056699 27409 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:48.057760 27272 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.839s	user 0.004s	sys 0.007s
I20260709 12:14:48.058274 27272 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9.
I20260709 12:14:48.057662 27384 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.059235 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 786) took 1088 ms. Trace:
I20260709 12:14:48.059625 23464 rpcz_store.cc:276] 0709 12:14:46.970518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.972836 (+  2318us) service_pool.cc:224] Handling call
0709 12:14:46.973231 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.973703 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.973746 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.984951 (+ 11205us) log.cc:844] Serialized 26 byte log entry
0709 12:14:47.284383 (+299432us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:47.333798 (+ 49415us) log.cc:844] Serialized 14 byte log entry
0709 12:14:47.361030 (+ 27232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.361138 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:47.470538 (+109400us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b480021c258. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:14:48.058921 (+588383us) raft_consensus.cc:1685] finished
0709 12:14:48.059017 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.059177 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26872,"prepare.queue_time_us":49436,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":5080,"raft.queue_time_us":99158,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":508401,"spinlock_wait_cycles":37273216,"thread_start_us":257694,"threads_started":1,"wal-append.queue_time_us":657295}
I20260709 12:14:47.852950 27253 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.063400 27253 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.063988 27253 raft_consensus.cc:2764] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:48.005726 27343 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:14:48.008353 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53209.5us
W20260709 12:14:47.943702 27385 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.493s	user 0.001s	sys 0.000s
I20260709 12:14:47.943055 27376 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.069867 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.070797 23321 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:48.071997 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:48.013456 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110172us
I20260709 12:14:48.021351 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:47.923934 27334 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:48.031229 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:48.108592 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77419.6us
I20260709 12:14:48.110343 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:48.111711 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 89933.5us
I20260709 12:14:47.853120 27387 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.032341 27401 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:48.036137 27348 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.230s	user 0.009s	sys 0.000s
I20260709 12:14:48.029672 27202 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.036688 27210 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.016644 27036 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:48.018077 27399 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.038698 27134 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:48.039623 26925 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.550s	user 0.012s	sys 0.000s
I20260709 12:14:48.017987 27244 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:48.145526 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 487655us
W20260709 12:14:48.040611 27294 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.001s
I20260709 12:14:48.099931 27360 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.008479 27296 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.167274 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:48.070501 27376 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.023525 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:48.073467 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.045372 23312 rpcz_store.cc:276] 0709 12:14:45.844887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.845024 (+   137us) service_pool.cc:224] Handling call
0709 12:14:45.845341 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:46.133501 (+288160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:46.133530 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:46.324654 (+191124us) log.cc:844] Serialized 26 byte log entry
0709 12:14:47.099584 (+774930us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:47.099625 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:47.099694 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.044583 (+944889us) raft_consensus.cc:1685] finished
0709 12:14:48.044603 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:48.044747 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":485044,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":24255,"thread_start_us":1009163,"threads_started":2,"wal-append.queue_time_us":884304}
I20260709 12:14:47.998688 27321 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:48.075057 27383 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.817s	user 0.000s	sys 0.001s
I20260709 12:14:48.012977 23315 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:14:47.922044 27394 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:47.871255 27341 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.081956 27243 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.063401 27375 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:48.083799 27318 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.086419 27254 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.011894 27413 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:48.093727 27203 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:48.094534 27311 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.000s
I20260709 12:14:48.096575 27382 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.098228 27161 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.099023 27247 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.121829 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:48.122632 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:48.124708 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:48.222123 23465 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:47.978046 27408 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:48.130290 23470 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:14:48.131865 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.223697 27412 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:14:48.224411 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.133045 23394 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:48.225621 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.226409 23321 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 6: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:48.227181 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.228258 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.228858 23321 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:48.167215 26925 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:48.217581 27413 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:48.232323 27413 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:48.145051 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:48.233319 23208 maintenance_manager.cc:419] P 259b912070e34dfba6c09de6bbb3d141: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033366
W20260709 12:14:48.108665 27209 consensus_meta.cc:287] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.934s	user 0.015s	sys 0.000s
I20260709 12:14:48.041034 27363 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:48.235941 23318 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:48.236503 23392 raft_consensus.cc:2368] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10.
I20260709 12:14:48.236929 27408 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.031477 27411 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:48.237388 27408 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.238502 27210 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.239178 27210 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:48.239986 27210 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:14:48.233364 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 88530.5us
I20260709 12:14:48.043195 27374 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:48.149281 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:48.170480 23436 leader_election.cc:400] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 10.
W20260709 12:14:48.172662 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 309968us
I20260709 12:14:48.241470 23980 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:48.186941 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:48.242584 23434 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.670s (SENT)
I20260709 12:14:48.128764 27223 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:48.241303 24013 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:48.243065 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:48.243253 27223 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.135748 27036 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.245631 27036 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:48.201870 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:48.146792 27395 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.249822 27395 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:48.250144 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 128401us
I20260709 12:14:48.250401 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.251209 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:48.251519 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:48.251996 27395 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:14:48.140090 27348 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.252679 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:48.255115 27348 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.254860 27311 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:48.256424 27348 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:48.256930 27348 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:48.139073 23359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.015s	sys 0.020s
W20260709 12:14:48.257632 23359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.016s	sys 0.020s
I20260709 12:14:48.259807 27416 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:48.261098 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.262283 27311 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:48.064708 27198 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.264878 27198 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.212880 27244 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:48.265775 27244 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.266244 27244 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:48.242875 23312 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:48.267513 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:48.268343 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:48.268959 23312 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:14:48.241654 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 71365.4us
W20260709 12:14:48.192220 27299 consensus_meta.cc:287] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.571s	user 0.007s	sys 0.006s
I20260709 12:14:48.262586 27223 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.265805 23467 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:48.133625 26702 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:48.252135 23314 raft_consensus.cc:1813] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Handling vote request from an unknown peer da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:48.362339 23314 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 10.
I20260709 12:14:48.240314 27210 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:48.278761 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.257314 27348 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:48.269773 27281 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.007s	sys 0.007s
I20260709 12:14:48.212195 27375 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:48.274462 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:48.344225 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:48.421137 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 227060us
I20260709 12:14:48.275556 23465 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:48.184582 27253 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
W20260709 12:14:48.352144 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 109586us
I20260709 12:14:48.436913 23321 raft_consensus.cc:2410] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:14:48.179548 27134 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:14:48.270211 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:48.181795 27294 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:48.356225 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 85668.9us
W20260709 12:14:48.358884 23361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.004s	sys 0.024s
I20260709 12:14:48.259826 24010 raft_consensus.cc:2368] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8.
I20260709 12:14:48.201922 27272 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 12:14:48.412775 27217 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:48.362049 26702 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:48.417321 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:48.418118 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 211960us
W20260709 12:14:48.420415 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:48.356467 27361 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:48.355433 27198 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:48.474251 27397 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.215s	user 0.007s	sys 0.000s
W20260709 12:14:48.475338 27425 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:14:48.221398 27418 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 12:14:48.417966 27417 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:14:48.479779 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43814.6us
I20260709 12:14:48.215500 27394 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.485445 23321 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
W20260709 12:14:48.496129 23361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.080s	user 0.005s	sys 0.024s
W20260709 12:14:48.270833 27338 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.948s	user 0.007s	sys 0.008s
W20260709 12:14:48.436480 27281 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.007s	sys 0.007s
I20260709 12:14:48.331483 27299 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:48.512989 23314 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:48.436328 27375 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:48.361958 27401 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.140s
I20260709 12:14:48.504920 27411 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:48.941808 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 440103us
W20260709 12:14:48.921546 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.022733 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1681) took 1514 ms. Trace:
W20260709 12:14:48.941648 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 447481us
I20260709 12:14:49.370744 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 919) took 1234 ms. Trace:
W20260709 12:14:49.094511 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.419780 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:49.090121 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 819927us
W20260709 12:14:48.941607 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 674160us
W20260709 12:14:49.169610 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:49.417526 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1683) took 1908 ms. Trace:
I20260709 12:14:49.419397 23315 rpcz_store.cc:276] 0709 12:14:47.508340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.508500 (+   160us) service_pool.cc:224] Handling call
0709 12:14:47.508932 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:48.190062 (+681130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.022353 (+832291us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400149454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:49.022651 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158255488}
W20260709 12:14:48.674782 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:49.329200 23312 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.060s	user 0.013s	sys 0.000s
I20260709 12:14:49.422963 23312 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:49.421743 23321 rpcz_store.cc:276] 0709 12:14:47.509020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.438700 (+929680us) service_pool.cc:224] Handling call
0709 12:14:48.472636 (+ 33936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.417176 (+944540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.417449 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.418774 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1692) took 1873 ms. Trace:
I20260709 12:14:49.424602 23314 rpcz_store.cc:276] 0709 12:14:47.545203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.512251 (+967048us) service_pool.cc:224] Handling call
0709 12:14:48.512692 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:49.418390 (+905698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.418712 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.428651 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:49.419591 24007 rpcz_store.cc:276] 0709 12:14:48.136858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.137041 (+   183us) service_pool.cc:224] Handling call
0709 12:14:48.137506 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:48.244452 (+106946us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:48.244482 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:48.244521 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:48.244553 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.370401 (+1125848us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54003741d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:49.370469 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.370651 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245491072}
I20260709 12:14:49.432631 23314 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:14:49.431061 23315 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:49.433210 23314 pending_rounds.cc:85] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
W20260709 12:14:49.434181 23312 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 978) took 1159 ms (client timeout 1506 ms). Trace:
I20260709 12:14:49.434508 23314 pending_rounds.cc:107] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.8
W20260709 12:14:49.436480 23312 rpcz_store.cc:269] 0709 12:14:48.264688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.268093 (+  3405us) service_pool.cc:224] Handling call
0709 12:14:49.423545 (+1155452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.436692 23314 raft_consensus.cc:2889] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:48.495761 27360 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:48.974375 27387 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.836s	user 0.000s	sys 0.009s
W20260709 12:14:49.298355 27414 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.244s	user 0.013s	sys 0.001s
W20260709 12:14:49.240885 27378 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.218s	user 0.008s	sys 0.004s
I20260709 12:14:49.437146 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1695) took 1887 ms. Trace:
W20260709 12:14:49.028110 27036 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.782s	user 0.006s	sys 0.000s
W20260709 12:14:48.944635 27337 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.065s	user 0.009s	sys 0.001s
I20260709 12:14:49.339188 27369 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:49.138873 27361 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:48.630524 27421 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:14:49.178002 27348 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.684s	user 0.009s	sys 0.003s
I20260709 12:14:48.513958 26702 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:48.631268 27287 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.370s	user 0.012s	sys 0.000s
W20260709 12:14:49.290879 27409 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.234s	user 0.007s	sys 0.005s
I20260709 12:14:49.142206 27224 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.345s
W20260709 12:14:49.029294 27408 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.791s	user 0.006s	sys 0.000s
W20260709 12:14:49.026136 27423 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.618s	user 0.000s	sys 0.001s
W20260709 12:14:49.048606 27336 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.362s	user 0.007s	sys 0.003s
W20260709 12:14:48.767145 27203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.042s	sys 0.077s
W20260709 12:14:49.021966 27433 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:14:49.431442 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:48.503338 27223 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:49.071617 27244 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.805s	user 0.001s	sys 0.006s
W20260709 12:14:49.441715 23979 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.006s	sys 0.007s
W20260709 12:14:49.525739 23979 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.008s	sys 0.010s
W20260709 12:14:49.437773 23981 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.003s	sys 0.010s
W20260709 12:14:49.526814 23981 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.008s	sys 0.011s
I20260709 12:14:49.527735 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:48.557211 27399 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:49.479221 23315 rpcz_store.cc:276] 0709 12:14:47.550207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.430591 (+1880384us) service_pool.cc:224] Handling call
0709 12:14:49.430904 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.436807 (+  5903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.437073 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:49.518247 27203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.264s	user 0.050s	sys 0.119s
W20260709 12:14:49.531822 23467 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 788) took 2553 ms (client timeout 2349 ms). Trace:
I20260709 12:14:49.474994 27414 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:49.532842 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 104340us
I20260709 12:14:49.257607 27430 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:49.529587 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.536027 23315 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.369119 27202 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.120050 27198 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:48.632279 27296 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.090881 27428 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:48.633105 27334 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.430661 27406 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:48.517721 27320 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.068410 27294 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:49.487476 23317 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
W20260709 12:14:49.479351 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.019s
I20260709 12:14:48.644389 27342 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:49.500701 27345 consensus_meta.cc:287] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.962s	user 0.008s	sys 0.007s
I20260709 12:14:49.438668 27360 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:49.483472 27420 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:14:49.474570 27416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.000s	sys 0.015s
I20260709 12:14:49.510157 24012 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:49.483695 27378 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:49.486370 27401 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:49.427309 27338 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:49.228341 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 22fc2096e78f47158bbb4bae3a474e2c with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43504
I20260709 12:14:49.502051 27348 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.572160 27447 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.517590 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 973) took 1349 ms. Trace:
I20260709 12:14:49.547883 27418 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.547706 27430 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:49.534276 27244 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:49.520215 24014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 918) took 2552 ms (client timeout 1632 ms). Trace:
I20260709 12:14:49.488448 27361 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:49.486585 27426 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.008s	sys 0.010s
W20260709 12:14:49.521121 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89737.6us
I20260709 12:14:49.557277 27420 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:49.485239 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 971) took 1632 ms. Trace:
I20260709 12:14:49.586958 23314 rpcz_store.cc:276] 0709 12:14:47.853090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.431886 (+1578796us) service_pool.cc:224] Handling call
0709 12:14:49.432186 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.484968 (+ 52782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.485178 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.477705 27387 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.545143 27202 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:14:48.739531 27384 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.534562 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:49.532233 23467 rpcz_store.cc:269] 0709 12:14:46.971136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.971262 (+   126us) service_pool.cc:224] Handling call
0709 12:14:49.521795 (+2550533us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400226fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:49.524987 (+  3192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2899428224}
I20260709 12:14:49.540050 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.606804 23313 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.488214 27036 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.486766 27436 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:14:49.541245 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.509639 27287 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.544706 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.546178 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.546816 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.613729 23468 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:49.582018 27430 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:49.512766 27409 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:49.529161 27337 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:14:49.548076 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.011s	sys 0.019s
W20260709 12:14:49.615718 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 673824us
I20260709 12:14:49.616693 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:49.618476 23982 leader_election.cc:400] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
W20260709 12:14:49.618731 27438 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:14:49.619266 23359 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.007s
W20260709 12:14:49.619627 23359 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.000s	sys 0.007s
I20260709 12:14:49.549949 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2441) took 2184 ms. Trace:
I20260709 12:14:49.549896 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2449) took 2162 ms. Trace:
I20260709 12:14:49.515072 27408 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.568626 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.621392 23388 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:49.621836 23466 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:49.520766 27336 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:49.623373 27437 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:14:49.574064 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.517522 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2442) took 2151 ms. Trace:
I20260709 12:14:49.441957 27304 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:14:49.533228 27223 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.579205 23316 rpcz_store.cc:276] 0709 12:14:48.167797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.508886 (+1341089us) service_pool.cc:224] Handling call
0709 12:14:49.516948 (+  8062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.517171 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:49.517209 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:49.517264 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:49.517311 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.517371 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.517517 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.554704 27360 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.581138 23391 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:49.581279 23319 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:14:48.639088 27376 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:49.582463 24014 rpcz_store.cc:269] 0709 12:14:46.958012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.958138 (+   126us) service_pool.cc:224] Handling call
0709 12:14:49.509700 (+2551562us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b540063e7d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:49.509874 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2139160960}
I20260709 12:14:49.585783 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:49.591326 23434 connection.cc:256] RPC call timeout handler was delayed by -1.1344126770000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
W20260709 12:14:49.648646 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0867126770001505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:14:49.583074 27361 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.598977 27414 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:49.599386 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 960) took 3777 ms (client timeout 2370 ms). Trace:
I20260709 12:14:49.542126 27198 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:49.574687 27401 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:48.741832 27210 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.588341 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:49.575975 27203 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:49.667897 27203 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:49.576512 27456 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.588797 27447 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.610906 23465 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:49.555611 27345 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:49.669742 27456 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.671187 27447 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:49.613094 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 78679us
I20260709 12:14:49.628049 27337 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:49.616801 27430 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:49.674240 23315 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:49.675632 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.675770 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:49.676677 24007 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:49.677165 27345 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.677702 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:49.677919 27387 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.643533 27451 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.678889 27451 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.679612 27341 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.530076 27399 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:49.682138 27399 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.650599 27281 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:49.683151 27399 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.683313 27281 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.686789 27036 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.686975 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:49.687760 23394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1790) took 1596 ms (client timeout 1636 ms). Trace:
I20260709 12:14:49.619537 27443 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:49.688957 27443 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:49.549408 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.689687 27408 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.625910 23387 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:49.620620 23390 rpcz_store.cc:276] 0709 12:14:47.389780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.506754 (+116974us) service_pool.cc:224] Handling call
0709 12:14:47.624678 (+117924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.637232 (+ 12554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.637271 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.669618 (+ 32347us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.017460 (+347842us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:48.017503 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.017571 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:48.062636 (+ 45065us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b4800002dd8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:14:49.544698 (+1482062us) raft_consensus.cc:1685] finished
0709 12:14:49.544723 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.549828 (+  5105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85622,"raft.run_cpu_time_us":1768,"raft.run_wall_time_us":342699,"spinlock_wait_cycles":9704704,"thread_start_us":248894,"threads_started":1,"wal-append.queue_time_us":565437}
I20260709 12:14:49.627900 23396 rpcz_store.cc:276] 0709 12:14:47.366287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.366429 (+   142us) service_pool.cc:224] Handling call
0709 12:14:47.366740 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.681776 (+315036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.681808 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.719105 (+ 37297us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.099935 (+380830us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:14:48.099976 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.100045 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:49.517238 (+1417193us) raft_consensus.cc:1685] finished
0709 12:14:49.517279 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.517458 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16960,"raft.queue_time_us":192248,"raft.run_cpu_time_us":4957,"raft.run_wall_time_us":41889,"thread_start_us":565439,"threads_started":2,"wal-append.queue_time_us":475261}
I20260709 12:14:49.620028 23389 rpcz_store.cc:276] 0709 12:14:47.365223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.365968 (+   745us) service_pool.cc:224] Handling call
0709 12:14:47.389123 (+ 23155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:47.683658 (+294535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:47.683686 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:47.814943 (+131257us) log.cc:844] Serialized 26 byte log entry
0709 12:14:48.012898 (+197955us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:14:48.012954 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:48.013091 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:49.549728 (+1536637us) raft_consensus.cc:1685] finished
0709 12:14:49.549752 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.549881 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26458,"raft.queue_time_us":284295,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11921,"thread_start_us":410081,"threads_started":2,"wal-append.queue_time_us":341389}
I20260709 12:14:49.625016 27336 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.635200 27299 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.601468 27458 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:49.607414 27455 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:14:49.646283 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:49.648067 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62363us
I20260709 12:14:49.583596 27418 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:49.653083 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 219004us
I20260709 12:14:49.709218 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:49.651650 23318 rpcz_store.cc:269] 0709 12:14:45.822186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.822630 (+   444us) service_pool.cc:224] Handling call
0709 12:14:49.599133 (+3776503us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b5400149e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:49.599287 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4910227584}
W20260709 12:14:49.656659 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 129014us
I20260709 12:14:49.658669 27406 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:49.663241 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2251) took 1585 ms. Trace:
I20260709 12:14:49.666023 27401 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.668838 27203 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.575922 27445 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:49.678297 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:49.688267 27244 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.690267 23470 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:14:49.556591 27416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.000s	sys 0.015s
I20260709 12:14:49.627907 27452 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:49.691618 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:49.688122 23394 rpcz_store.cc:269] 0709 12:14:48.091829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.686676 (+1594847us) service_pool.cc:224] Handling call
0709 12:14:49.687702 (+  1026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.701684 27281 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.692739 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:49.692209 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.693502 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:49.772894 23315 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:14:49.776474 27317 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.813s	user 0.009s	sys 0.001s
W20260709 12:14:49.777657 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65721us
I20260709 12:14:49.778319 27348 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:49.778751 27224 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:49.779749 27443 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.772886 23468 raft_consensus.cc:2368] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:49.618924 23982 leader_election.cc:403] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:49.695118 23313 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:49.642400 27411 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.696945 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:49.783020 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73240.3us
I20260709 12:14:49.654179 27253 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:49.609608 27384 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.713114 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:49.787493 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:49.788033 23394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1786) took 1697 ms (client timeout 1995 ms). Trace:
W20260709 12:14:49.788259 23394 rpcz_store.cc:269] 0709 12:14:48.090673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.771809 (+1681136us) service_pool.cc:224] Handling call
0709 12:14:49.787990 (+ 16181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.735036 27336 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:49.735859 27036 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:49.789898 24007 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:14:49.737053 23362 connection.cc:256] RPC call timeout handler was delayed by -1.1931918970001334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:14:49.717418 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:49.776978 27317 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.792018 23467 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.712095 23320 rpcz_store.cc:276] 0709 12:14:48.078026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.487696 (+1409670us) service_pool.cc:224] Handling call
0709 12:14:49.511129 (+ 23433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.511282 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:49.658131 (+146849us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.662474 (+  4343us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.662773 (+   299us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:49.662834 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:49.662911 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.663000 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.663164 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7091,"prepare.queue_time_us":81662,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":10178,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":24316,"spinlock_wait_cycles":8399104,"thread_start_us":141357,"threads_started":2,"wal-append.queue_time_us":36236}
I20260709 12:14:49.690742 27337 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:49.792982 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.793617 23318 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:14:49.760700 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45395.7us
W20260709 12:14:49.761552 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83344.1us
I20260709 12:14:49.769577 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:49.692652 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.783186 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 970) took 1930 ms. Trace:
I20260709 12:14:49.783383 24016 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.681133 27451 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:49.784718 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 166439us
I20260709 12:14:49.795467 23462 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:14:49.800233 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:49.780762 27452 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.801077 23470 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.801659 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.655334 27420 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.806447 27420 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:49.736544 27345 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.795578 23396 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:49.784920 27253 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.760571 27398 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:49.769260 27445 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.789835 27462 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:14:49.806329 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:49.768217 27408 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:49.761034 27296 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:14:49.706342 26702 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.808621 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:49.798990 27411 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:49.795837 23321 rpcz_store.cc:276] 0709 12:14:47.852828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.431193 (+1578365us) service_pool.cc:224] Handling call
0709 12:14:49.431463 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.431636 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:14:49.519224 (+ 87588us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.782846 (+263622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:49.782901 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:49.782950 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.783021 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:49.783135 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":685,"prepare.queue_time_us":144215,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":1713,"raft.queue_time_us":1176,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6344,"thread_start_us":342576,"threads_started":2}
I20260709 12:14:49.784297 27456 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:49.789219 27401 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:49.813982 27253 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:49.711006 27458 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.979969 27465 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.812479 27338 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.807628 27452 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:49.794199 27336 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:49.836861 27345 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.908507 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.987954 23320 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:14:49.988759 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:49.992411 27321 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:49.799854 27036 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.910133 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:49.997277 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 87201.6us
I20260709 12:14:49.998346 23362 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:49.818368 27461 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:14:49.959213 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 169173us
I20260709 12:14:50.006736 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:49.963670 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1791) took 1871 ms. Trace:
I20260709 12:14:50.007680 23394 rpcz_store.cc:276] 0709 12:14:48.092103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.788735 (+1696632us) service_pool.cc:224] Handling call
0709 12:14:49.963592 (+174857us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:49.809808 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1788) took 1718 ms (client timeout 1942 ms). Trace:
W20260709 12:14:50.008613 23396 rpcz_store.cc:269] 0709 12:14:48.091253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.711571 (+1620318us) service_pool.cc:224] Handling call
0709 12:14:49.809751 (+ 98180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:49.807473 27281 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:49.972922 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:49.910609 27451 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.819950 27406 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:50.012389 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.013085 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:49.917670 27455 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:14:49.953815 27304 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.975358 27408 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:49.980571 27465 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:49.954236 27414 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:14:49.988539 27412 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:50.044994 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46756.5us
I20260709 12:14:50.045785 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:50.034546 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 225125us
I20260709 12:14:50.036192 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.047681 23466 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:50.032007 23321 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:50.048463 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 974) took 1880 ms. Trace:
I20260709 12:14:50.048615 23470 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:50.048851 23321 rpcz_store.cc:276] 0709 12:14:48.168178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.025749 (+1857571us) service_pool.cc:224] Handling call
0709 12:14:50.026146 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.048190 (+ 22044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.048401 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:50.050094 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2455) took 2577 ms. Trace:
I20260709 12:14:50.017802 27428 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:50.008347 27223 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:49.981709 27411 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:50.051430 23982 connection.cc:256] RPC call timeout handler was delayed by -1.3182767470000272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 12:14:50.017509 27334 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:50.031064 27464 tablet_replica.cc:333] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:14:50.050477 23391 rpcz_store.cc:276] 0709 12:14:47.472833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.060446 (+587613us) service_pool.cc:224] Handling call
0709 12:14:48.060751 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.644562 (+1583811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.049872 (+405310us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002d1ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:50.050039 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3496211840}
I20260709 12:14:50.067139 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.032117 27409 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.044602 27473 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:50.114185 23394 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:49.947778 27398 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.044080 27296 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.115939 27455 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:14:50.019687 27281 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.086256 27287 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.046126 27304 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.132712 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.163513 23391 raft_consensus.cc:2376] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10. Re-sending same reply.
W20260709 12:14:50.165208 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 105583us
I20260709 12:14:50.166721 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:50.155663 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2143) took 3050 ms (client timeout 1533 ms). Trace:
I20260709 12:14:50.156023 23464 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:50.032095 27451 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.157379 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:50.174885 23318 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:50.176589 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 712) took 2338 ms. Trace:
I20260709 12:14:50.181622 24012 rpcz_store.cc:276] 0709 12:14:47.817945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.871192 (+ 53247us) service_pool.cc:224] Handling call
0709 12:14:49.509973 (+1638781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.556766 (+ 46793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.156186 (+599420us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540063e7d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.156363 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3768160128}
W20260709 12:14:50.167760 23463 rpcz_store.cc:269] 0709 12:14:47.105676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.239266 (+133590us) service_pool.cc:224] Handling call
0709 12:14:50.155398 (+2916132us) spinlock_profiling.cc:243] Waited 971 ms on lock 0x7b5400186fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.155569 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2232781568}
I20260709 12:14:50.184196 27478 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.184986 27478 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.188874 27478 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.190302 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:50.087003 27334 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:14:50.201265 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44009.9us
I20260709 12:14:50.202739 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.210100 23391 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:50.211199 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.211862 23391 raft_consensus.cc:2376] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10. Re-sending same reply.
I20260709 12:14:50.212901 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.214397 24007 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:50.215865 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:50.216601 24007 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:50.218042 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.218219 24011 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:50.219512 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.220896 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.052881 27412 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.126875 27473 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:50.231534 27299 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.140333 27445 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:50.232499 23982 leader_election.cc:336] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.686s (SENT)
I20260709 12:14:50.267441 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:50.268332 23320 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
W20260709 12:14:50.268805 23982 leader_election.cc:336] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.865s (SENT)
I20260709 12:14:50.232879 24009 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:14:50.232906 27223 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.233757 24015 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:14:50.270942 24015 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:50.273808 24007 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:50.202173 27414 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.221684 27365 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:14:50.246387 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:50.231850 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.284111 24008 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:50.249688 24016 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:50.251235 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.252024 23394 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:50.286265 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.222677 27334 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:50.259811 27286 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:50.250875 27463 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.466s	user 0.001s	sys 0.000s
I20260709 12:14:50.236194 23467 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:50.271674 24015 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.293388 24015 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:50.295003 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:50.250991 27384 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.276372 27338 raft_consensus.cc:2764] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:14:50.318075 27465 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.321178 24015 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:14:50.345693 24015 pending_rounds.cc:85] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 9
I20260709 12:14:50.345978 24015 pending_rounds.cc:107] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:14:50.346201 24015 raft_consensus.cc:2889] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:50.320981 27452 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.313843 23466 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:50.322427 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.277109 26925 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.142098 27296 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:50.296283 27261 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 2.864s	user 0.030s	sys 0.000s
W20260709 12:14:50.338006 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.349488 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 303753us
I20260709 12:14:50.377274 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 979) took 2112 ms. Trace:
I20260709 12:14:50.377900 23318 rpcz_store.cc:276] 0709 12:14:48.264941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.191700 (+1926759us) service_pool.cc:224] Handling call
0709 12:14:50.192116 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.192395 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:50.322840 (+130445us) log.cc:844] Serialized 14 byte log entry
0709 12:14:50.376768 (+ 53928us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:50.376848 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:50.376935 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.377033 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.377194 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11573,"thread_start_us":171358,"threads_started":2}
I20260709 12:14:50.350126 23320 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:50.155056 27408 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:50.353502 27369 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.862s	user 0.011s	sys 0.008s
I20260709 12:14:50.293649 27435 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:50.275514 27299 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.394896 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:50.387388 27452 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:50.392719 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 226119us
I20260709 12:14:50.395005 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:50.324332 27458 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.397403 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.398551 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:50.398679 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.292394 27223 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.400660 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2250) took 2322 ms. Trace:
W20260709 12:14:50.407107 23982 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.965s (SENT)
I20260709 12:14:50.406500 23312 rpcz_store.cc:276] 0709 12:14:48.077766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.483946 (+1406180us) service_pool.cc:224] Handling call
0709 12:14:49.484317 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:49.484660 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:49.484690 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:49.492088 (+  7398us) log.cc:844] Serialized 52 byte log entry
0709 12:14:49.606842 (+114754us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:49.615212 (+  8370us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.633775 (+ 18563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:49.633897 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:50.400429 (+766532us) raft_consensus.cc:1685] finished
0709 12:14:50.400450 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.400598 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33466,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":7097,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":23049,"spinlock_wait_cycles":11140480,"thread_start_us":78620,"threads_started":1,"wal-append.queue_time_us":180510}
I20260709 12:14:50.402667 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:50.412973 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.414565 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.415305 23466 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.416788 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.418282 23467 raft_consensus.cc:2410] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:14:50.419536 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.022s	sys 0.016s
W20260709 12:14:50.419927 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.022s	sys 0.016s
W20260709 12:14:50.396512 27337 consensus_meta.cc:287] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.585s	user 0.015s	sys 0.046s
I20260709 12:14:50.389308 27036 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:50.404989 27399 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.017s
W20260709 12:14:50.410667 27361 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.725s	user 0.019s	sys 0.014s
W20260709 12:14:50.422008 27375 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.982s	user 0.016s	sys 0.012s
W20260709 12:14:50.422278 27430 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.007s
I20260709 12:14:50.402163 27336 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.410079 27458 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:50.357100 27473 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.381916 27261 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.513495 27459 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 12:14:50.613730 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:50.614614 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 154299us
W20260709 12:14:50.656586 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.396378 27287 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:14:50.465235 27475 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:14:50.616571 24011 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:14:50.614542 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 213231us
I20260709 12:14:50.617188 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 981) took 1089 ms. Trace:
I20260709 12:14:50.624913 27452 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:50.399984 26925 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:50.620792 23466 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:14:50.621312 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 199561us
I20260709 12:14:50.660000 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.660290 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:50.660748 23466 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:50.661528 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.662077 27445 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:50.493904 27411 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.614285 27320 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:50.662621 27445 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:50.662997 27320 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.663527 27445 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.662115 23466 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:50.627246 27483 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.238s	user 0.004s	sys 0.001s
W20260709 12:14:50.627548 27341 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.947s	user 0.015s	sys 0.000s
W20260709 12:14:50.628427 27253 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.016s
W20260709 12:14:50.628255 27462 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.646s	user 0.013s	sys 0.000s
I20260709 12:14:50.384714 27455 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:50.661581 23982 leader_election.cc:336] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 2.458s (SENT)
I20260709 12:14:50.622177 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:50.674892 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61123.7us
W20260709 12:14:50.676239 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.676584 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.677892 23461 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:50.678429 23466 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:50.679955 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:50.626338 27337 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.685098 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.679780 23469 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:50.638680 27361 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.687458 23463 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:50.665868 26925 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:50.666559 27458 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:50.688236 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.674288 27281 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.643065 27375 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.681107 27462 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:50.688336 27294 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.684725 27490 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:50.689888 26925 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.681267 27253 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.647921 27430 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.696691 27299 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.705578 27490 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:50.709079 27401 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.728s	user 0.004s	sys 0.009s
I20260709 12:14:50.711884 27451 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:50.709759 27492 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.082s	user 0.001s	sys 0.001s
I20260709 12:14:50.706640 27412 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:50.709466 27360 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.055s	user 0.004s	sys 0.007s
I20260709 12:14:50.713878 27458 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.721053 27359 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:50.712113 27490 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:50.736408 27428 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.003s
I20260709 12:14:50.736999 27428 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:14:50.728762 27420 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.922s	user 0.017s	sys 0.007s
I20260709 12:14:50.725206 27359 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:50.738945 27359 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.740410 27359 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.741633 27361 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.725325 27360 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.746212 27360 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:50.780695 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 122196us
I20260709 12:14:50.783623 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:50.658432 23318 rpcz_store.cc:276] 0709 12:14:49.527337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.403994 (+876657us) service_pool.cc:224] Handling call
0709 12:14:50.616647 (+212653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.616800 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.616823 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:50.616860 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:50.616987 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.617045 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.617139 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:50.794891 23314 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:50.797279 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2452) took 3349 ms. Trace:
I20260709 12:14:50.684109 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.799250 23390 raft_consensus.cc:2368] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11.
I20260709 12:14:50.799898 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 687) took 1127 ms. Trace:
I20260709 12:14:50.800259 23390 rpcz_store.cc:276] 0709 12:14:49.672732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.683850 (+1011118us) service_pool.cc:224] Handling call
0709 12:14:50.799775 (+115925us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:50.806353 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.806088 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.807909 23466 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:50.797885 23393 rpcz_store.cc:276] 0709 12:14:47.447519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.042526 (+595007us) service_pool.cc:224] Handling call
0709 12:14:48.042878 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:48.043200 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:48.043225 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:50.276263 (+2233038us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:14:50.299175 (+ 22912us) log.cc:844] Serialized 26 byte log entry
0709 12:14:50.421668 (+122493us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:50.421715 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.421813 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:50.796295 (+374482us) raft_consensus.cc:1685] finished
0709 12:14:50.796366 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.796934 (+   568us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22559,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":182000512,"thread_start_us":47729,"threads_started":1,"wal-append.queue_time_us":342582}
W20260709 12:14:50.810971 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2445) took 3309 ms (client timeout 2203 ms). Trace:
I20260709 12:14:50.807824 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.818274 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.809144 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.826442 27223 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.828043 27244 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:50.829807 27253 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.704119 27294 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.709934 27296 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.832635 27412 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:50.833822 27430 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.839381 27375 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.868772 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:50.866456 23392 rpcz_store.cc:269] 0709 12:14:47.369203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.369372 (+   169us) service_pool.cc:224] Handling call
0709 12:14:50.678547 (+3309175us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.678681 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1726417152}
I20260709 12:14:50.883690 26925 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.886389 24012 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:50.886787 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.887945 23390 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.890333 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:50.891216 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.896466 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.897450 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:50.897217 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.898825 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.900895 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.901919 23463 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:50.902606 27317 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:50.892500 23394 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:50.903720 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:50.893918 27337 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:50.894323 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 708) took 3917 ms (client timeout 2203 ms). Trace:
W20260709 12:14:50.909402 24010 rpcz_store.cc:269] 0709 12:14:46.976428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:46.976754 (+   326us) service_pool.cc:224] Handling call
0709 12:14:50.893820 (+3917066us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003cb754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.894063 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2948055936}
I20260709 12:14:50.905072 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.911492 23394 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:50.915257 27334 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.001s
I20260709 12:14:50.916190 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.901904 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.890342 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:50.916987 23396 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:50.902697 27261 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.892282 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 797) took 1091 ms. Trace:
I20260709 12:14:50.892148 26925 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:50.905007 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.917865 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 985) took 1135 ms. Trace:
I20260709 12:14:50.929664 23392 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:50.930256 27304 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:50.931258 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.932189 23392 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:50.919180 23462 rpcz_store.cc:276] 0709 12:14:49.800789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.801446 (+   657us) service_pool.cc:224] Handling call
0709 12:14:50.891906 (+1090460us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54001072d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.892072 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1371156352}
I20260709 12:14:50.933367 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.934387 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1801) took 1120 ms. Trace:
I20260709 12:14:50.935678 23392 rpcz_store.cc:276] 0709 12:14:49.813681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.932941 (+1119260us) service_pool.cc:224] Handling call
0709 12:14:50.934338 (+  1397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:50.936930 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:50.940856 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:50.942147 23392 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:50.942806 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1796) took 1145 ms. Trace:
I20260709 12:14:50.945703 23392 rpcz_store.cc:276] 0709 12:14:49.796998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.936580 (+1139582us) service_pool.cc:224] Handling call
0709 12:14:50.942731 (+  6151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:50.948501 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:50.926136 23320 rpcz_store.cc:276] 0709 12:14:49.782541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.404861 (+622320us) service_pool.cc:224] Handling call
0709 12:14:50.478680 (+ 73819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.614595 (+135915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.614628 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:50.615183 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:14:50.912086 (+296903us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:50.912149 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.912257 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:50.917323 (+  5066us) raft_consensus.cc:1685] finished
0709 12:14:50.917342 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:50.917603 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1230,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":223297,"threads_started":1,"wal-append.queue_time_us":187722}
I20260709 12:14:50.976052 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1680) took 3468 ms. Trace:
I20260709 12:14:50.976563 23319 rpcz_store.cc:276] 0709 12:14:47.507774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.507996 (+   222us) service_pool.cc:224] Handling call
0709 12:14:47.624478 (+116482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:49.644910 (+2020432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.975443 (+1330533us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b540001e354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:50.975857 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4499712384}
W20260709 12:14:50.978685 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:50.979811 23317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1684) took 3470 ms (client timeout 2050 ms). Trace:
I20260709 12:14:50.915868 27334 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:50.981585 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:50.984479 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:50.980376 23317 rpcz_store.cc:269] 0709 12:14:47.509322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.966361 (+457039us) service_pool.cc:224] Handling call
0709 12:14:50.979288 (+3012927us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002418d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:50.979507 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3184590208}
I20260709 12:14:50.985915 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:50.986796 27415 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:50.987913 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:50.987768 27415 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:50.988888 23463 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:50.986842 23393 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:14:50.995126 23390 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:50.996490 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:50.997277 27432 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 1.451s	user 0.004s	sys 0.003s
I20260709 12:14:51.004658 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:51.006384 27418 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.004889 27317 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:51.069481 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.069952 27334 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.070432 27412 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.072805 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:51.084626 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.084213 27412 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:51.085268 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.088074 27412 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:51.093967 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89658.7us
I20260709 12:14:51.094327 27224 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:51.095006 27224 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:51.096487 27224 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.099236 27415 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.101316 27382 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.102355 23390 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.102293 27382 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:51.104869 23390 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:14:51.108446 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 790) took 2994 ms. Trace:
I20260709 12:14:51.108870 23464 rpcz_store.cc:276] 0709 12:14:48.114260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.115229 (+   969us) service_pool.cc:224] Handling call
0709 12:14:50.155752 (+2040523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.198861 (+ 43109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.108074 (+909213us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400186fd0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.108268 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4691936512}
I20260709 12:14:51.110451 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.110610 27382 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.111240 23464 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:51.007865 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.113422 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 688) took 1440 ms. Trace:
I20260709 12:14:51.113791 23392 rpcz_store.cc:276] 0709 12:14:49.672976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.949701 (+1276725us) service_pool.cc:224] Handling call
0709 12:14:51.113339 (+163638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.115319 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2447) took 3744 ms. Trace:
I20260709 12:14:51.118733 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.119593 24015 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:14:51.122349 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:51.115381 23392 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 5. Current term is 6. Ops: []
I20260709 12:14:51.123517 27202 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:51.123833 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1787) took 3032 ms. Trace:
I20260709 12:14:51.123917 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1785) took 3033 ms. Trace:
I20260709 12:14:51.124490 23390 rpcz_store.cc:276] 0709 12:14:48.090381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.103473 (+3013092us) service_pool.cc:224] Handling call
0709 12:14:51.104511 (+  1038us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:51.105256 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.123822 (+ 18566us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.125775 27401 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:51.126235 27401 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:51.127594 27401 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:51.124935 27202 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:51.129698 27202 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.130112 27202 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:51.082326 23394 raft_consensus.cc:2410] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:14:51.129171 27401 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:51.138743 23396 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:51.138878 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1798) took 1335 ms. Trace:
I20260709 12:14:51.124300 23392 rpcz_store.cc:276] 0709 12:14:48.090964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.114480 (+3023516us) service_pool.cc:224] Handling call
0709 12:14:51.115021 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.123565 (+  8544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.123759 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.141754 23394 rpcz_store.cc:276] 0709 12:14:49.803104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.904704 (+1101600us) service_pool.cc:224] Handling call
0709 12:14:51.138796 (+234092us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.124138 27486 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:51.147151 27486 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:51.147462 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:51.148535 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.149303 23396 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:51.148969 27486 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.151312 27210 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:51.151634 27486 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:51.115803 23395 rpcz_store.cc:276] 0709 12:14:47.370770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.370896 (+   126us) service_pool.cc:224] Handling call
0709 12:14:47.387588 (+ 16692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:47.816804 (+429216us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:48.077630 (+260826us) log.cc:844] Serialized 14 byte log entry
0709 12:14:49.043567 (+965937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:49.043651 (+    84us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:49.043722 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.114797 (+2071075us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54000a8f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:51.114886 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.115147 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266382,"prepare.queue_time_us":404810,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":835460,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":22050,"spinlock_wait_cycles":986530048,"thread_start_us":382928,"threads_started":2,"wal-append.queue_time_us":425395,"wal-append.run_cpu_time_us":37688,"wal-append.run_wall_time_us":2061697}
I20260709 12:14:51.161399 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.167024 23396 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:51.170468 23395 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:14:51.171170 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 691) took 1497 ms. Trace:
I20260709 12:14:51.171500 23395 rpcz_store.cc:276] 0709 12:14:49.673679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.158530 (+1484851us) service_pool.cc:224] Handling call
0709 12:14:51.170006 (+ 11476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.170910 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.171107 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.173017 23395 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:51.174162 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1793) took 1377 ms. Trace:
I20260709 12:14:51.174512 23395 rpcz_store.cc:276] 0709 12:14:49.796268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.172286 (+1376018us) service_pool.cc:224] Handling call
0709 12:14:51.172720 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.173406 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.174076 (+   670us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:51.175917 27484 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.477s	user 0.002s	sys 0.007s
I20260709 12:14:51.176002 23395 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:51.177330 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1794) took 1380 ms. Trace:
I20260709 12:14:51.177855 23395 rpcz_store.cc:276] 0709 12:14:49.796511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.175387 (+1378876us) service_pool.cc:224] Handling call
0709 12:14:51.175792 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.177073 (+  1281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.177270 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:51.178826 27488 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.860s	user 0.003s	sys 0.004s
I20260709 12:14:51.178400 27434 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:51.179306 23395 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:51.183118 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1795) took 1386 ms. Trace:
I20260709 12:14:51.183671 23394 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:51.184455 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 690) took 1511 ms. Trace:
I20260709 12:14:51.184824 23394 rpcz_store.cc:276] 0709 12:14:49.673484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.144663 (+1471179us) service_pool.cc:224] Handling call
0709 12:14:51.183390 (+ 38727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:51.184189 (+   799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.184376 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220992}
I20260709 12:14:51.164891 23393 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:51.188349 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 803) took 1375 ms. Trace:
I20260709 12:14:51.188783 23470 rpcz_store.cc:276] 0709 12:14:49.811203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.065377 (+254174us) service_pool.cc:224] Handling call
0709 12:14:50.070788 (+  5411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.215818 (+145030us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.216067 (+   249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:50.217027 (+   960us) log.cc:844] Serialized 26 byte log entry
0709 12:14:50.394019 (+176992us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:14:50.394058 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.394127 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:51.174873 (+780746us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:51.185755 (+ 10882us) raft_consensus.cc:1685] finished
0709 12:14:51.185796 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.186142 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29,"raft.queue_time_us":149551,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":118,"spinlock_wait_cycles":11132800,"thread_start_us":176498,"threads_started":1,"wal-append.queue_time_us":466581}
W20260709 12:14:51.191731 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.192725 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.194232 23394 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:14:51.193123 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2154) took 1646 ms. Trace:
I20260709 12:14:51.198468 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1802) took 1250 ms. Trace:
I20260709 12:14:51.199038 23394 rpcz_store.cc:276] 0709 12:14:49.947980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.185680 (+1237700us) service_pool.cc:224] Handling call
0709 12:14:51.186110 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.198077 (+ 11967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.198342 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:51.200726 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.183454 23395 rpcz_store.cc:276] 0709 12:14:49.796768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.178710 (+1381942us) service_pool.cc:224] Handling call
0709 12:14:51.179080 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.182856 (+  3776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.183054 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.198098 23469 rpcz_store.cc:276] 0709 12:14:49.546126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.546267 (+   141us) service_pool.cc:224] Handling call
0709 12:14:51.192754 (+1646487us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.192940 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2419479808}
I20260709 12:14:51.209839 23464 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:14:51.217540 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.219586 27416 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:51.220263 27416 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:51.220654 27416 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.226830 23392 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:51.227573 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 689) took 1554 ms. Trace:
I20260709 12:14:51.227903 23392 rpcz_store.cc:276] 0709 12:14:49.673217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.145184 (+1471967us) service_pool.cc:224] Handling call
0709 12:14:51.205224 (+ 60040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.227307 (+ 22083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.227503 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.219831 23315 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:14:51.219699 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 975) took 3017 ms. Trace:
I20260709 12:14:51.235917 27202 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.257615 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.258605 24014 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:51.259758 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.229125 27317 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.272468 27321 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:51.273049 27321 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:51.272620 27317 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.274355 27321 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.275753 27321 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:51.276223 27317 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.277547 23389 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:51.278992 27198 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:51.280006 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68461.4us
I20260709 12:14:51.284045 23395 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:51.285094 23389 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:51.285792 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1806) took 1064 ms. Trace:
I20260709 12:14:51.297807 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.301986 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.303372 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.287091 23395 rpcz_store.cc:276] 0709 12:14:50.220415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.213135 (+992720us) service_pool.cc:224] Handling call
0709 12:14:51.269262 (+ 56127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.284561 (+ 15299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.284947 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.309734 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:51.313005 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.307565 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.229904 23321 rpcz_store.cc:276] 0709 12:14:48.180740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.050568 (+1869828us) service_pool.cc:224] Handling call
0709 12:14:50.051000 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:50.246084 (+195084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:50.246115 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:50.246568 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:14:50.259015 (+ 12447us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:50.259067 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:50.259162 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:51.177931 (+918769us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:51.185185 (+  7254us) raft_consensus.cc:1685] finished
0709 12:14:51.185216 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.197989 (+ 12773us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8134,"raft.queue_time_us":277400,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":9745,"spinlock_wait_cycles":19882752,"thread_start_us":198641,"threads_started":2,"wal-append.queue_time_us":51045}
I20260709 12:14:51.322623 23392 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:14:51.324154 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1807) took 1059 ms. Trace:
I20260709 12:14:51.324568 23392 rpcz_store.cc:276] 0709 12:14:50.265174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.261315 (+996141us) service_pool.cc:224] Handling call
0709 12:14:51.309146 (+ 47831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.323167 (+ 14021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.324091 (+   924us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:51.329969 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.330914 23315 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:51.331038 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:51.332365 24015 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:51.333693 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.336650 27345 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.343025 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.349819 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.354714 27202 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.355881 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.358384 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:51.360920 23393 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
W20260709 12:14:51.374218 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.384536 27434 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:14:51.412747 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1789) took 3318 ms. Trace:
I20260709 12:14:51.413230 23390 rpcz_store.cc:276] 0709 12:14:48.091540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.125799 (+3034259us) service_pool.cc:224] Handling call
0709 12:14:51.126258 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.126717 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:14:51.166339 (+ 39622us) log.cc:844] Serialized 14 byte log entry
0709 12:14:51.409454 (+243115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:51.409517 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:51.409578 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.409646 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.409778 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14,"prepare.queue_time_us":179755,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":7540,"raft.queue_time_us":4288,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"thread_start_us":232515,"threads_started":2}
I20260709 12:14:51.416536 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:51.419106 27516 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:14:51.409602 23434 leader_election.cc:336] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.632s (SENT)
I20260709 12:14:51.426983 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:51.411959 23982 leader_election.cc:336] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 3.078s (SENT)
I20260709 12:14:51.431191 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:51.437062 23391 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:51.437492 27320 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.437837 27477 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:51.438723 23391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 686) took 1766 ms (client timeout 1965 ms). Trace:
I20260709 12:14:51.438572 27477 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:51.439221 23391 rpcz_store.cc:269] 0709 12:14:49.672548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.246957 (+574409us) service_pool.cc:224] Handling call
0709 12:14:51.438394 (+1191437us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002c7254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.438618 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2645485824}
I20260709 12:14:51.443331 24008 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:51.446066 27466 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:14:51.448644 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74774us
I20260709 12:14:51.430389 27416 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.448758 27434 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.453747 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.456898 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1803) took 1368 ms. Trace:
I20260709 12:14:51.457270 23394 rpcz_store.cc:276] 0709 12:14:50.085784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.206235 (+1120451us) service_pool.cc:224] Handling call
0709 12:14:51.283713 (+ 77478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:51.342745 (+ 59032us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:14:51.345675 (+  2930us) log.cc:844] Serialized 14 byte log entry
0709 12:14:51.452475 (+106800us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:51.452528 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:14:51.452576 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.453373 (+   797us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.454009 (+   636us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30597,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":1848,"thread_start_us":104525,"threads_started":1}
I20260709 12:14:51.462021 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:51.467586 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:51.465262 23434 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.691s (SENT)
I20260709 12:14:51.469884 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:51.472867 23463 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:51.466045 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.474136 23319 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:14:51.475801 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:51.473923 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 83623.9us
I20260709 12:14:51.478374 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:51.485772 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:51.488598 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.492280 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.491983 27466 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:51.492982 24016 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:14:51.489733 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.852s (SENT)
I20260709 12:14:51.497581 24008 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:51.501927 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:51.504096 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:51.504668 27448 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 1.967s	user 0.000s	sys 0.012s
I20260709 12:14:51.505652 24013 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:51.507268 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2446) took 4131 ms. Trace:
I20260709 12:14:51.510738 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:51.512241 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:51.509045 23387 rpcz_store.cc:276] 0709 12:14:47.369497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:47.369627 (+   130us) service_pool.cc:224] Handling call
0709 12:14:47.369874 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:49.698001 (+2328127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.480871 (+1782870us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54001964d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:51.501309 (+ 20438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5187899776}
I20260709 12:14:51.513774 24008 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:51.516043 24016 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:51.519294 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:51.522001 27466 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:51.527043 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 927) took 1320 ms. Trace:
I20260709 12:14:51.527478 24011 rpcz_store.cc:276] 0709 12:14:50.206597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.237500 (+ 30903us) service_pool.cc:224] Handling call
0709 12:14:50.237834 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.657732 (+419898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.526633 (+868901us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400650ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:51.526866 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509860736}
W20260709 12:14:51.542857 27510 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 12:14:51.546180 23388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 682) took 3902 ms (client timeout 2349 ms). Trace:
W20260709 12:14:51.546728 23388 rpcz_store.cc:269] 0709 12:14:47.642842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:48.023214 (+380372us) service_pool.cc:224] Handling call
0709 12:14:51.545118 (+3521904us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54002d23d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.545361 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3303976320}
I20260709 12:14:51.547358 27376 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.547950 27376 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.550191 27376 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:51.551179 27519 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:14:51.541004 23287 connection.cc:256] RPC call timeout handler was delayed by -2.4898812430001271s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 12:14:51.552737 23287 connection.cc:256] RPC call timeout handler was delayed by -2.4601812430000791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 12:14:51.553272 23287 connection.cc:256] RPC call timeout handler was delayed by -2.1784812430000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 12:14:51.553591 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9426812430001519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 12:14:51.553891 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9057812430000922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 12:14:51.553860 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1792) took 1757 ms. Trace:
W20260709 12:14:51.554529 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9057812430000922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 12:14:51.554653 23396 rpcz_store.cc:276] 0709 12:14:49.796004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:51.197617 (+1401613us) service_pool.cc:224] Handling call
0709 12:14:51.217400 (+ 19783us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:51.298474 (+ 81074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:51.298526 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:51.299427 (+   901us) log.cc:844] Serialized 52 byte log entry
0709 12:14:51.425088 (+125661us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:51.425137 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.425216 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:51.553558 (+128342us) raft_consensus.cc:1685] finished
0709 12:14:51.553587 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:51.553783 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44681,"raft.queue_time_us":46612,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":205,"thread_start_us":125165,"threads_started":1,"wal-append.queue_time_us":163720}
I20260709 12:14:51.556704 26702 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:51.557914 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68534.4us
W20260709 12:14:51.558763 23434 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.890s (SENT)
I20260709 12:14:51.561512 27416 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.561918 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.562526 27456 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:51.562924 27456 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:51.563598 23464 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:14:51.564118 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.564057 27456 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.564846 23468 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:51.564975 23466 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:14:51.563607 24007 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:51.566661 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.567377 23467 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:51.568077 27473 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:51.565690 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:51.570016 24011 raft_consensus.cc:2410] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:14:51.571424 24011 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:14:51.572201 27477 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:51.573930 23287 connection.cc:256] RPC call timeout handler was delayed by -1.8679812430000311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260709 12:14:51.574960 27456 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:51.578315 27200 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:51.587157 23467 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:51.590072 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:51.591064 23466 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.593799 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:51.595113 27439 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 2.069s	user 0.000s	sys 0.008s
I20260709 12:14:51.595669 23465 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:51.600172 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.600584 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.600920 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.601449 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2255) took 2076 ms. Trace:
I20260709 12:14:51.601811 23394 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:51.601856 23314 rpcz_store.cc:276] 0709 12:14:49.525833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.587990 (+ 62157us) service_pool.cc:224] Handling call
0709 12:14:51.601131 (+2013141us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54002742d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.601340 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2590119168}
I20260709 12:14:51.600966 23388 raft_consensus.cc:2376] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11. Re-sending same reply.
I20260709 12:14:51.605273 27398 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.593856 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:51.606534 27436 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:51.608578 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.616026 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 795) took 1815 ms. Trace:
I20260709 12:14:51.616590 23461 rpcz_store.cc:276] 0709 12:14:49.800200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.800826 (+   626us) service_pool.cc:224] Handling call
0709 12:14:49.801070 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:50.682850 (+881780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:51.615565 (+932715us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54002da0d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:51.615832 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016113536}
I20260709 12:14:51.621968 27294 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.623308 27394 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.626287 27394 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:14:51.627293 27512 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.271s	user 0.003s	sys 0.000s
I20260709 12:14:51.627650 27321 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:51.628064 27321 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:51.630748 24012 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 920) took 1426 ms (client timeout 1738 ms). Trace:
W20260709 12:14:51.631173 24012 rpcz_store.cc:269] 0709 12:14:50.204332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:50.246118 (+ 41786us) service_pool.cc:224] Handling call
0709 12:14:51.630517 (+1384399us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b5400775e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:51.630675 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":954059136}
I20260709 12:14:51.632539 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.632961 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.633580 23388 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.633762 23393 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:14:51.634140 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.634385 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.634902 23390 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.634795 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.635711 23392 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.635551 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.625699 27411 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:51.636771 27411 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.638335 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.638967 27411 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.633785 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.640144 27436 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:51.640774 27436 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.642863 27436 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:51.636226 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.643642 23389 raft_consensus.cc:2368] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11.
I20260709 12:14:51.629113 27321 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:51.638964 23392 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.645750 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.646407 23469 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:51.647689 23465 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:51.650015 23464 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:51.651108 27394 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:51.644963 27321 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.601634 23387 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:51.572227 27520 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.098s	user 0.001s	sys 0.001s
I20260709 12:14:51.653108 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:51.662456 23389 raft_consensus.cc:606] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:51.654502 27464 raft_consensus.cc:2243] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:51.663154 23389 raft_consensus.cc:614] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:51.665016 24016 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:51.666824 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:51.666923 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.721673 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.667379 23394 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:14:51.624106 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.652231 23392 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:51.667541 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.667783 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.668304 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.669272 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.670415 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:51.671766 27475 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:51.672501 27464 pending_rounds.cc:70] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Trying to abort 1 pending ops.
I20260709 12:14:51.671830 27394 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:14:51.673941 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.727218 23389 raft_consensus.cc:606] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:51.731487 23389 raft_consensus.cc:614] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:51.682487 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:51.683491 27321 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:14:51.684836 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:51.735004 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50245.9us
I20260709 12:14:51.686650 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:51.736123 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 124887us
I20260709 12:14:51.736799 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:51.696182 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.730600 27394 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:51.729185 27464 pending_rounds.cc:77] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305622488421662720 op_type: NO_OP noop_request { }
I20260709 12:14:51.738348 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.738637 27464 raft_consensus.cc:2889] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:51.739074 27464 raft_consensus.cc:2272] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:51.739567 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.697386 27401 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:14:51.697723 27398 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:51.697356 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.742338 23320 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:51.743261 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.743661 27394 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:51.744748 23320 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:51.698984 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.700008 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.701170 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.702426 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.726125 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:51.726648 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:51.713088 24008 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:14:51.729362 23314 raft_consensus.cc:2410] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:51.727178 23391 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
W20260709 12:14:51.748263 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:51.750057 23319 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:51.752447 23319 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:51.752889 23313 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:51.752470 23321 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:51.755122 23434 leader_election.cc:336] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.969s (SENT)
I20260709 12:14:51.756047 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:51.756690 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.754122 27436 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:51.760866 23396 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:51.753712 27376 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:51.762279 27394 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:14:51.783380 27376 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:51.765712 23320 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:14:51.798924 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 269669us
I20260709 12:14:51.781952 27411 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:51.806308 23217 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:51.807276 27466 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:51.809528 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:51.810742 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64592.5us
I20260709 12:14:51.811028 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:51.811906 23396 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:51.812386 23396 raft_consensus.cc:614] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:14:51.814122 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81456.5us
W20260709 12:14:51.815835 23434 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.027s (SENT)
I20260709 12:14:51.816495 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:51.819525 27477 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:51.820103 27477 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:51.820977 27477 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:51.809580 27161 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:51.829998 27464 ts_tablet_manager.cc:1916] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:14:51.830229 27161 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:51.830754 27161 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:51.808785 23217 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 6, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:51.800627 27475 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:51.853353 23321 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:51.854377 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.855208 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:51.869144 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 71447.2us
I20260709 12:14:51.871081 23392 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.871906 23389 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:51.874328 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:51.871964 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 201837us
I20260709 12:14:51.877476 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.851269 27466 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:51.869144 23217 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:51.881924 23391 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:51.881938 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.885113 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:14:51.886775 23470 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:51.883860 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:51.889194 23387 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:51.894661 23389 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:51.895555 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.896136 23392 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:14:51.883154 27480 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.228s	user 0.010s	sys 0.019s
I20260709 12:14:51.850592 27376 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:51.901676 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:51.904762 23321 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:14:51.906319 23391 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:51.932835 23319 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:51.934700 23314 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:51.935200 23314 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:51.936354 23314 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:51.901232 27480 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.393s	user 0.010s	sys 0.019s
I20260709 12:14:51.962023 23317 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:51.967554 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:52.002256 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:52.005834 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102029us
I20260709 12:14:52.018285 23314 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.018492 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:52.021420 23463 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:14:52.005281 23982 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.565s (SENT) [suppressed 41 similar messages]
I20260709 12:14:52.018196 26702 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:14:52.030148 23396 raft_consensus.cc:606] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:52.030589 23396 raft_consensus.cc:614] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:52.003755 27466 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:14:52.037081 23394 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:52.061034 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 245243us
W20260709 12:14:52.063962 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.037s (SENT)
I20260709 12:14:52.064428 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.065157 27401 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:52.054400 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.086354 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:14:52.087193 23396 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:52.087677 23396 raft_consensus.cc:614] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:52.012529 27161 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.056293 27466 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:52.095345 23392 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:14:52.106122 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.081586 27321 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:14:52.124070 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60172.2us
I20260709 12:14:52.127583 23314 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:14:52.135739 27348 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:14:52.138507 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:52.141223 23287 leader_election.cc:400] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10.
I20260709 12:14:52.141637 23287 leader_election.cc:403] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:14:52.148970 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48118us
I20260709 12:14:52.155731 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:52.157457 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.157253 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:52.158018 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.166764 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:52.153393 27456 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:52.168306 27466 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.171104 27449 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:52.171643 27449 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.172025 27449 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:52.172551 27465 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:52.173090 27465 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:52.173615 27434 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:52.174435 27490 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:52.174908 27200 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:52.154803 27480 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10.
W20260709 12:14:52.176554 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.171172 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:52.179670 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:52.190467 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:52.200296 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 143356us
I20260709 12:14:52.198026 23467 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:14:52.177882 27394 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:52.205245 27394 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.171176 27348 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 12:14:52.200404 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 167354us
I20260709 12:14:52.200585 27436 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:52.205672 27394 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:52.247897 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68325.7us
I20260709 12:14:52.197970 27321 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 12:14:52.251536 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 78870.3us
I20260709 12:14:52.275137 27485 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:52.275743 27485 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.276134 27485 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:52.278517 27537 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:14:52.279446 27542 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:52.279804 27409 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.280325 27409 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:52.283142 26702 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:52.282495 27409 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:52.284631 27161 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:52.217952 27416 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:14:52.246043 27486 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:52.295883 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.324031 27532 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.324808 27465 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.326033 27536 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:52.348408 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 96197.8us
I20260709 12:14:52.353245 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:52.367019 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:52.367695 23313 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:52.335861 27409 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:14:52.379120 27411 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:52.379515 27411 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:52.380499 27411 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:52.355705 27536 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.384968 27536 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:52.400952 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 259916us
W20260709 12:14:52.416294 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 168621us
I20260709 12:14:52.418361 27542 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:52.419924 27542 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:52.420413 27449 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.420850 27542 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.424650 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:52.425642 27554 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:52.427515 27485 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:52.415534 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62369.2us
I20260709 12:14:52.448253 23313 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:14:52.450888 27554 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.451695 27485 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:52.452320 27485 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:52.452648 27485 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:52.453032 27485 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:52.454777 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:52.449751 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:52.461047 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 289922us
I20260709 12:14:52.468508 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:52.469960 24008 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:52.351648 27532 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.489874 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:52.496143 23317 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:52.499886 24012 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:52.497714 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:52.508087 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:52.466358 27161 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:52.523178 27538 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.274s	user 0.016s	sys 0.009s
I20260709 12:14:52.529242 27224 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:52.529863 27224 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:52.530700 27409 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:52.531199 27409 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:52.531636 27409 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.531054 27224 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.532532 27224 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:52.532315 27161 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:52.534646 27532 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:52.466358 27465 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:14:52.539471 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43778.1us
I20260709 12:14:52.542177 24008 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:52.546484 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:52.542160 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46185.1us
I20260709 12:14:52.548192 23313 raft_consensus.cc:2410] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:52.548427 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:52.550709 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:52.551887 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:52.554071 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.556027 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:52.556356 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.515475 27553 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:52.562433 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:52.565390 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:52.569747 23468 pending_rounds.cc:85] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 7
I20260709 12:14:52.576723 23319 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:14:52.577554 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.570228 23468 pending_rounds.cc:107] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:14:52.586323 23468 raft_consensus.cc:2889] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:52.588186 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.592515 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:52.593358 23318 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:52.603094 23982 leader_election.cc:336] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.745s (SENT)
I20260709 12:14:52.603773 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.584340 27536 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:52.621119 27536 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:52.622313 27536 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:52.617892 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.591003 27401 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:52.577405 27394 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:52.658219 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:52.655076 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 86513.9us
I20260709 12:14:52.646871 27200 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:52.657985 27411 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:14:52.675369 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 78416.1us
I20260709 12:14:52.680101 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:52.646526 27480 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:14:52.730830 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:52.760119 23982 leader_election.cc:336] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.646s (SENT)
W20260709 12:14:52.788672 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102467us
I20260709 12:14:52.791711 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:52.662333 27533 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:14:52.800148 23464 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:52.680998 27573 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:52.686890 27378 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:52.862039 23287 leader_election.cc:400] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11.
I20260709 12:14:52.862524 23287 leader_election.cc:403] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:52.752864 27553 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:52.884558 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72710.9us
W20260709 12:14:52.886493 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 126421us
I20260709 12:14:52.799906 27338 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:14:52.820688 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:52.890311 27338 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:52.761546 27456 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 12:14:52.790647 27321 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 12:14:52.865271 27573 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:52.865813 27378 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:52.940410 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53127.7us
I20260709 12:14:52.934490 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:52.890476 27547 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11.
I20260709 12:14:52.942003 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:52.949234 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 400867us
I20260709 12:14:52.933420 27338 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.919481 27449 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:52.957710 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:52.940052 27378 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.939920 27573 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:52.965090 27553 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:52.967085 27563 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
W20260709 12:14:52.965183 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 79871.3us
W20260709 12:14:52.972215 23434 leader_election.cc:336] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.636s (SENT)
W20260709 12:14:52.959223 27562 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 12:14:52.983817 23387 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:14:52.985729 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:52.990315 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:14:52.999419 23982 leader_election.cc:336] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.851s (SENT)
W20260709 12:14:53.003048 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 52770us
I20260709 12:14:53.003847 27394 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:53.004196 27570 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:14:53.005635 27394 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:53.006834 27394 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:53.008693 27394 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.011286 27549 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:14:53.011912 27549 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:53.012070 27547 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:53.012681 27547 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:53.013628 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.014087 27566 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:14:53.013731 27547 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.014497 23314 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:53.015362 27561 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 12:14:53.015681 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.016822 27569 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:14:53.016958 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.017808 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.017889 27338 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.018787 27579 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:53.019174 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.020226 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.020973 27338 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.021835 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:53.026500 27485 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.570s	user 0.010s	sys 0.000s
I20260709 12:14:53.026968 27485 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.022040 23318 raft_consensus.cc:2410] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:53.030138 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.030794 23313 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.039124 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.039876 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:53.040791 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.015369 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.021252 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.029162 27485 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:53.012987 27549 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.044737 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.044975 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45586.5us
I20260709 12:14:53.066131 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.066406 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94221.8us
I20260709 12:14:53.047573 27161 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:53.067324 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.048846 27401 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:53.049549 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.055538 23317 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:53.055131 23313 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:14:53.057770 27224 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.002s
I20260709 12:14:53.078855 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.068056 23312 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:53.080940 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.082362 23315 raft_consensus.cc:2393] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:53.084410 23321 raft_consensus.cc:2368] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8.
I20260709 12:14:53.090011 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.090690 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.093214 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.105028 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.009249 27394 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.109261 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.114344 23314 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:53.115902 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:14:53.116628 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:53.121932 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.148844 23318 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:53.149467 23318 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:53.153003 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44656.4us
I20260709 12:14:53.157709 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.174825 24016 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:14:53.165948 27409 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.634s	user 0.020s	sys 0.003s
I20260709 12:14:53.167883 27588 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.182267 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:53.172502 27464 ts_tablet_manager.cc:1929] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.4
I20260709 12:14:53.183279 27586 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:53.185194 27464 log.cc:1199] T 22fc2096e78f47158bbb4bae3a474e2c P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/22fc2096e78f47158bbb4bae3a474e2c
I20260709 12:14:53.185895 23318 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.190078 27477 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:53.198343 27477 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:14:53.202096 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.210840 27464 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43504 (request call id 0) took 4238 ms. Trace:
I20260709 12:14:53.211323 27464 rpcz_store.cc:276] 0709 12:14:49.047188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:49.090814 (+ 43626us) service_pool.cc:224] Handling call
0709 12:14:49.809624 (+718810us) ts_tablet_manager.cc:1151] Deleting tablet 22fc2096e78f47158bbb4bae3a474e2c
0709 12:14:49.953893 (+144269us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:14:52.428799 (+2474906us) tablet_metadata.cc:665] Metadata flushed
0709 12:14:53.169046 (+740247us) tablet_metadata.cc:665] Metadata flushed
0709 12:14:53.210737 (+ 41691us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1532235,"tablet-delete.queue_time_us":206349,"thread_start_us":124718,"threads_started":1}
I20260709 12:14:53.219522 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:53.221284 23387 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:53.182242 27409 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.229128 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:53.235325 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:53.235898 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:53.236178 24012 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:53.236348 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:53.236692 24010 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:53.236946 24009 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:53.239326 27579 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:53.242983 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:53.246981 24009 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:14:53.202572 27401 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:53.251819 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:53.252784 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.255214 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.876s (SENT)
I20260709 12:14:53.255798 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.185349 27586 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.187798 27588 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.261969 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.273540 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.280947 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.281303 27588 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.290009 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.290743 23467 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:53.295004 23470 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:53.202085 27477 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.266050 27466 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:53.309242 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.268887 27579 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:53.324481 27579 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.327248 27555 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:14:53.260517 27572 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:14:53.318449 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.333212 23461 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.320653 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.323089 27409 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.316437 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.341729 23463 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:53.343708 27520 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.694s	user 0.008s	sys 0.008s
I20260709 12:14:53.331959 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:53.344977 24008 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:53.349018 23463 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:53.339453 27574 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:53.368055 27574 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:53.304872 27321 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.334614 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.340314 27378 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.350291 23466 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:53.376544 23470 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.376771 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.385843 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.387008 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.387436 27348 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:53.387814 23466 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
W20260709 12:14:53.377684 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68015.7us
I20260709 12:14:53.390126 27449 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:53.390511 23982 leader_election.cc:400] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:53.393241 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.378815 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.394732 23314 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:53.393649 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.393921 23469 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:53.397116 23467 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.400856 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:53.401422 24009 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:53.404095 23470 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:53.407191 24010 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:14:53.373237 27394 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.409263 27532 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:53.409735 27532 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:53.416263 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.417510 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.418506 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.419427 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.420570 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:53.421285 24014 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:53.423535 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.275676 27401 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.392287 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.425904 23316 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 4.
I20260709 12:14:53.426811 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:53.393669 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.428570 23463 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:14:53.425465 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:53.430871 23316 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:53.432385 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.432493 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.432971 24009 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:53.433646 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.434669 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.439538 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:53.440260 23321 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:53.373517 27586 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.447554 23315 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:14:53.450466 23394 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:53.441895 23319 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:14:53.461285 23467 raft_consensus.cc:2410] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:14:53.466858 23434 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.942s (SENT)
I20260709 12:14:53.473250 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.474820 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:53.434034 27394 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.469686 23320 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:53.470311 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.479053 23316 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:53.475009 23434 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.995s (SENT)
I20260709 12:14:53.476013 27600 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:53.482036 27597 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:14:53.476912 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.384393 27321 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.487174 27321 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:53.491091 27564 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.813s	user 0.003s	sys 0.000s
I20260709 12:14:53.495030 27401 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:53.497370 27394 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.499531 27394 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:53.502118 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:53.502934 23396 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:53.477270 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:53.506503 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 119485us
I20260709 12:14:53.477403 23467 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.467501 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.514144 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:53.472870 27572 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.481619 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:53.518054 23392 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:53.517905 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:53.518783 23395 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:53.466699 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:53.506843 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:53.520136 23387 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:53.510227 24009 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:53.512584 23396 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:53.522404 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:53.479928 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.476805 27466 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.524000 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 97293.5us
I20260709 12:14:53.528402 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:53.530449 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:53.531788 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 95747.1us
I20260709 12:14:53.533142 23467 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:53.395102 27574 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.535718 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:53.530122 23388 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:53.558118 23362 connection.cc:256] RPC call timeout handler was delayed by -1.7626917580000736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
W20260709 12:14:53.558799 23362 connection.cc:256] RPC call timeout handler was delayed by -1.7347917580000285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
W20260709 12:14:53.559127 23362 connection.cc:256] RPC call timeout handler was delayed by -1.694291757999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20600000000000002s for a response
W20260709 12:14:53.559358 23362 connection.cc:256] RPC call timeout handler was delayed by -1.319789566000054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
W20260709 12:14:53.559585 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2990895660000206s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 12:14:53.559794 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2414895659999274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 12:14:53.560025 23362 connection.cc:256] RPC call timeout handler was delayed by -1.1973895659998561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 12:14:53.560241 23362 connection.cc:256] RPC call timeout handler was delayed by -1.0065895660000024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 12:14:53.531159 27409 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:53.531530 27456 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:53.513581 27401 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:53.525130 27348 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:53.569183 23395 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:53.548867 27559 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.573303 23362 leader_election.cc:400] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:53.568531 27574 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:53.576117 27552 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.583316 27348 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.586295 27378 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:53.526523 27572 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.575515 23362 leader_election.cc:403] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:14:53.561264 27594 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:53.566883 27589 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:53.598248 23389 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:14:53.531422 27466 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.570896 27456 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:53.618024 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47837.5us
I20260709 12:14:53.625649 27600 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:53.611145 27394 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:14:53.626758 27394 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.627187 27394 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:53.608714 27594 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.629658 27589 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:53.630146 27589 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:53.577108 27552 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.608969 27378 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10.
I20260709 12:14:53.571859 27409 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.612273 27456 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:53.641659 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.660187 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.668322 27589 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.670305 27589 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:14:53.672688 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 197726us
W20260709 12:14:53.673595 23434 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.050s (SENT)
I20260709 12:14:53.674086 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.678694 27401 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:53.686843 27401 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:53.687988 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.639087 27594 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:53.711076 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 174921us
W20260709 12:14:53.691226 23434 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.127s (SENT)
W20260709 12:14:53.749780 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 136440us
I20260709 12:14:53.753387 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:53.753716 23287 connection.cc:256] RPC call timeout handler was delayed by -1.3188573500001439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 12:14:53.754333 23287 connection.cc:256] RPC call timeout handler was delayed by -1.3098573500001294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
W20260709 12:14:53.755016 23287 connection.cc:256] RPC call timeout handler was delayed by -1.3062573500001236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 12:14:53.710575 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 185901us
W20260709 12:14:53.755437 23287 connection.cc:256] RPC call timeout handler was delayed by -1.2891573500000959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 12:14:53.755909 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1784573500001443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
W20260709 12:14:53.756311 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1451573500000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 12:14:53.756906 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1451573500000904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 12:14:53.757318 23287 connection.cc:256] RPC call timeout handler was delayed by -1.144257350000089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
W20260709 12:14:53.757723 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1316573500000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 12:14:53.758106 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1172573500000453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 12:14:53.758507 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0920573500000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
W20260709 12:14:53.697777 27561 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:14:53.712401 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:53.610402 27559 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.763545 27594 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:14:53.783200 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 143748us
I20260709 12:14:53.785125 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:53.783602 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 71308.6us
I20260709 12:14:53.786036 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:53.786437 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.788554 23461 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:53.768576 27585 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.795519 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.804641 27616 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:53.805208 27616 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.805603 27616 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:53.807157 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:53.808743 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.809674 27477 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:53.810431 27477 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:53.811615 27477 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.809724 27594 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:53.817965 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 126779us
W20260709 12:14:53.819725 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.794442 27378 raft_consensus.cc:2764] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:14:53.822813 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:53.797251 27549 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:14:53.830724 27610 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:14:53.833581 27554 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:53.835170 27613 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:53.835587 27613 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:53.835906 27613 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:53.832999 27559 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:53.839054 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:53.840363 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.841147 27558 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:53.841804 27558 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:53.842743 27558 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.845654 27594 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:53.850466 27552 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:53.841243 27573 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.003s
I20260709 12:14:53.822844 27621 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:14:53.832980 27394 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.794332 27585 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:53.886562 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.886945 27436 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:53.897466 27436 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:53.897929 27436 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:53.903225 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 99122.2us
I20260709 12:14:53.854213 27549 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:53.895735 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 109774us
I20260709 12:14:53.910738 27549 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:53.894239 27572 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:53.917856 27559 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:53.921635 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.924216 27394 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:53.938860 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 168200us
I20260709 12:14:53.887593 27456 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:53.887189 27573 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:53.970541 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:53.980494 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:53.969484 27409 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:53.961840 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 141567us
I20260709 12:14:52.958343 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:53.967002 23217 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:53.980211 27598 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:53.996294 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.361s (SENT)
I20260709 12:14:53.997052 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:53.990888 27559 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:14:54.018337 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 100634us
W20260709 12:14:54.019860 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.074517 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:54.076210 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63654.8us
I20260709 12:14:54.001173 27598 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:54.085176 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 99913.2us
I20260709 12:14:54.084170 27394 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:54.089994 27394 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.092100 27394 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.094327 27573 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.099298 23464 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:54.100289 27598 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.101184 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.101804 24014 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 6.
I20260709 12:14:54.076151 27401 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.109822 27401 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.110878 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.112037 27401 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.112241 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.085880 27589 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:14:54.112486 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.112947 24010 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:54.113201 27589 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:14:54.113596 27589 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:54.113198 24014 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:54.113992 24012 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:54.114523 27589 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.115762 27589 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.115926 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.120115 27585 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:54.086664 27436 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.120721 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.125003 24015 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:54.126685 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.131124 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:54.133383 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:54.097671 27637 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:14:54.144438 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47171.4us
I20260709 12:14:54.148300 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.149283 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.150043 24013 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:54.150986 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:54.134356 27585 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:54.154587 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:54.162531 23982 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.545s (SENT)
W20260709 12:14:54.166328 23982 leader_election.cc:336] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.565s (SENT)
I20260709 12:14:54.166729 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:54.168619 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.153554 27577 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.160492 27637 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:54.210209 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 135811us
I20260709 12:14:54.145212 27616 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.241146 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:54.242450 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69788.8us
I20260709 12:14:54.219784 26702 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:54.247562 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 78563.7us
I20260709 12:14:54.248545 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:54.249684 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:54.261776 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95475.3us
W20260709 12:14:54.266112 27634 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:14:54.266603 27532 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.268136 27532 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:54.273556 27600 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:54.276244 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.282181 27640 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:54.287916 23389 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:14:54.290297 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49277.4us
I20260709 12:14:54.291932 23390 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:54.292440 23394 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:54.293453 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.294235 23387 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:54.291940 23395 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
W20260709 12:14:54.301770 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.229913 27637 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:54.302349 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.302616 27637 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.303509 23395 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:54.305060 23389 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:54.305706 27616 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:54.313671 27622 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.480s	user 0.009s	sys 0.000s
I20260709 12:14:54.314795 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.315805 23394 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:54.317093 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56444.7us
I20260709 12:14:54.317963 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.287714 27613 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.319418 27573 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:54.321107 27613 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:54.339124 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 74601.8us
I20260709 12:14:54.311793 27466 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:54.321369 23394 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:54.307534 27436 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:54.341943 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.326220 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.345216 23390 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:14:54.347946 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.349243 27600 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:54.349793 27600 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.350217 27600 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.352003 27588 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:54.353575 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62304.3us
I20260709 12:14:54.354293 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.355697 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.356621 27436 raft_consensus.cc:2764] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:54.357298 27436 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:54.358117 27436 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:54.358511 27436 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:54.358891 27436 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:54.328112 27637 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:54.329488 27549 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.363579 27637 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:54.364049 27637 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:54.364894 27549 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:54.331399 27640 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:54.365876 27640 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:54.366190 27640 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:54.365031 27637 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.366940 27640 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.367808 27637 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:54.368103 27640 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.332156 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.333056 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:54.333698 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84083.3us
I20260709 12:14:54.374842 27559 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:54.375317 27559 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.375752 27559 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:54.377490 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45412.6us
W20260709 12:14:54.378201 23982 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.790s (SENT)
I20260709 12:14:54.378630 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.378780 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.379456 27556 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:54.380064 27556 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.380473 27556 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.382527 27556 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:54.384359 27558 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.385478 27436 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:54.336068 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.337589 23395 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:54.258842 27585 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:54.327631 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.386063 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:54.433480 23391 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
W20260709 12:14:54.389565 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55976.4us
I20260709 12:14:54.391724 27456 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.399642 23387 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:14:54.402724 27559 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:54.445804 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59789.5us
I20260709 12:14:54.447794 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.334079 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.438557 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.435660 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:54.436558 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:54.448838 23389 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:54.449164 24010 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:14:54.440934 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.449517 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.442526 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.377223 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:54.450028 23391 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.451880 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:54.455479 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.461369 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:54.462177 24012 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.493059 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:54.495692 23393 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:14:54.496735 23468 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:54.498514 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.492532 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.502442 23395 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:54.511186 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 134057us
W20260709 12:14:54.512552 23982 leader_election.cc:336] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.703s (SENT)
I20260709 12:14:54.513015 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.461966 27586 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:14:54.463346 27648 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:14:54.492280 27409 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:54.511780 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.464157 27573 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:54.510308 27224 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.459372 27579 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:54.512081 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.459600 27616 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:54.519233 27556 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:54.521103 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:54.532065 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.532689 23389 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
W20260709 12:14:54.514370 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 158731us
I20260709 12:14:54.521507 27409 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:14:54.462952 27580 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:54.538467 24012 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:54.530309 27586 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:54.531337 27585 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:14:54.534540 27224 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.559975 27224 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:54.549396 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.549751 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.566160 27586 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.566773 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.566988 27598 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.549902 27589 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.557946 27532 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:54.567934 27600 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.568655 27559 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.566814 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:54.570358 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.571493 27598 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:54.571929 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.572172 27598 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:54.572885 27598 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:54.573704 27647 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:54.544569 27409 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:54.574241 27647 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.568338 27532 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.559227 27579 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.570457 27600 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.542779 27558 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:14:53.222759 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 22fc2096e78f47158bbb4bae3a474e2c (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:14:54.586385 24010 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:54.590035 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.590970 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:54.591744 24015 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:54.588369 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 127723us
I20260709 12:14:54.570360 27549 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.606784 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.609511 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:54.610653 23393 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.577113 27585 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:54.612876 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.614912 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:54.615558 23467 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:54.615795 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.548506 27580 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.576361 27647 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.618687 27647 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:54.619328 27647 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:14:54.621423 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:54.587347 27559 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:54.635036 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:14:54.645979 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:54.645300 23206 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 1.42256e+06us
I20260709 12:14:54.649894 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:54.650537 23469 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 4.
I20260709 12:14:54.612665 27321 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:54.645942 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:54.616945 27585 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:54.686474 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:54.622699 27580 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:14:54.712512 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.715976 23318 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:54.690469 23393 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:54.646396 27475 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:54.722538 23320 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:14:54.652482 27613 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.590903 27579 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.657804 27588 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:54.625571 27486 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:54.621645 27549 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.730913 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.722945 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.682864 27436 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.729390 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:54.746136 23314 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:54.750540 23313 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:14:54.750888 23313 pending_rounds.cc:85] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:14:54.751096 23313 pending_rounds.cc:107] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:14:54.751448 23313 raft_consensus.cc:2889] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.746506 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:54.746526 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:54.749025 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.691740 27321 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:54.689575 27434 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:14:54.761551 23317 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
W20260709 12:14:54.763033 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:54.765843 27617 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.916s	user 0.003s	sys 0.000s
I20260709 12:14:54.766016 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:54.765718 27616 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:54.770867 23312 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:14:54.780547 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.620s (SENT)
I20260709 12:14:54.787822 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:54.789145 23434 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.636s (SENT)
W20260709 12:14:54.786924 23982 leader_election.cc:336] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.715s (SENT)
I20260709 12:14:54.731612 27652 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:54.789815 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:54.791507 23318 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 11. (term mismatch)
I20260709 12:14:54.792001 23318 pending_rounds.cc:85] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:14:54.792297 23318 pending_rounds.cc:107] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.11
I20260709 12:14:54.792600 23318 raft_consensus.cc:2889] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:54.793538 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2292) took 1748 ms. Trace:
W20260709 12:14:54.795001 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:54.800630 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 233928us
I20260709 12:14:54.657685 27559 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.817377 23469 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:54.745798 27475 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:54.794246 23318 rpcz_store.cc:276] 0709 12:14:53.045479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:53.109138 (+ 63659us) service_pool.cc:224] Handling call
0709 12:14:53.148431 (+ 39293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:54.644651 (+1496220us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b580076cdd0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4
0709 12:14:54.793153 (+148502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:54.793459 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487331072}
W20260709 12:14:54.803709 27480 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:14:54.807158 27416 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:54.744215 27613 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.806171 27600 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.804111 27652 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:54.751147 27486 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:54.842890 27649 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:14:54.864108 27613 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.869552 27321 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.871260 27549 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.874063 27588 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:54.875718 23468 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:54.883561 27348 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:54.891160 27224 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.895129 27559 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.900926 23391 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:54.903990 23395 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:14:54.906155 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73380us
W20260709 12:14:54.912271 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.937s (SENT)
I20260709 12:14:54.920560 24015 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:54.920956 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:54.913975 27588 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:54.914915 27348 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:54.859404 27652 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:54.917182 27532 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.953981 27600 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:54.959265 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47065.9us
I20260709 12:14:54.922329 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.919529 27409 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:54.927351 27657 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:54.926868 27348 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:54.966229 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43970.5us
I20260709 12:14:54.967900 27532 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:54.968210 27579 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:54.969986 27409 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:54.949203 27654 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:54.974092 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:54.974428 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 251574us
I20260709 12:14:54.981129 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:54.975471 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:54.925797 27559 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 12:14:54.978533 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 96289.9us
W20260709 12:14:54.988981 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:54.978293 27532 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:54.996456 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 231036us
I20260709 12:14:54.928124 27475 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.006928 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.007694 23395 raft_consensus.cc:2368] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:54.992650 27549 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:14:55.009073 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.013760 23395 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:55.013495 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.015322 23391 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:55.010704 23217 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:55.017073 23391 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:55.019825 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.020478 23389 raft_consensus.cc:2376] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11. Re-sending same reply.
I20260709 12:14:55.012363 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.025096 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.000497 27532 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.024310 27657 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:14:55.021816 27549 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.033026 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.033954 23387 raft_consensus.cc:2410] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:14:55.033893 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.037744 23312 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:14:55.008997 27224 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.005748 27659 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:14:54.995980 27585 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.046326 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.037273 27434 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.047207 23313 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:55.041317 23388 raft_consensus.cc:2368] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8.
I20260709 12:14:55.042475 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.035006 27549 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:14:55.040807 27573 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.027s	sys 0.008s
I20260709 12:14:54.993530 27654 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.053659 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.054559 27559 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:55.053654 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.047097 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.072582 23391 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14. Re-sending same reply.
I20260709 12:14:55.040575 27616 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:55.074159 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:55.077838 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47967.4us
I20260709 12:14:55.079267 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.080081 27654 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.080353 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.080686 27654 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:55.086349 23387 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:55.043467 27465 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:55.089895 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.090420 27436 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:55.091467 27637 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.723s	user 0.012s	sys 0.001s
I20260709 12:14:55.072280 27625 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:14:55.092912 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.093412 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.094165 23388 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:55.094935 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.095618 23388 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:14:55.096561 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.070943 27659 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.097601 27588 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.098652 27659 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.090620 23394 raft_consensus.cc:2368] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:55.102299 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:14:55.095768 27640 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.727s	user 0.003s	sys 0.008s
I20260709 12:14:55.104322 27640 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:55.104691 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.105471 27656 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:14:55.070228 27573 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.057508 27434 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.106053 27670 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:55.106820 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.107522 27434 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.108042 23470 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:14:55.105748 27616 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.105916 27656 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.106530 27573 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.112423 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.112607 27579 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:55.112963 27579 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.113042 23470 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:55.113631 27670 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:55.115103 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.115242 27436 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:14:55.115831 27436 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.116694 27436 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.115290 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.119123 24015 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:14:55.117348 23393 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:14:55.115813 23392 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:55.121904 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.122066 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.122767 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.039773 27558 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:14:55.125771 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:55.126331 24014 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:55.120874 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.118600 23470 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:55.123989 23468 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:14:55.125231 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.036005 27348 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:55.136211 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.141044 23468 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:55.144106 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.146508 23464 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:55.145572 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:55.147449 24015 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:14:55.149886 23469 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:14:55.051712 27661 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:14:55.126570 27640 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:55.162427 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.170382 23462 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:14:55.174392 23465 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:14:55.174901 23465 pending_rounds.cc:85] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 4
I20260709 12:14:55.175174 23465 pending_rounds.cc:107] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:14:55.175472 23465 raft_consensus.cc:2889] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:55.177289 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.177961 23462 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.181583 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.126705 27637 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:14:55.192308 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.193151 23464 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:55.182102 27348 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:55.206151 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.185348 27668 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:55.220891 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:55.267584 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61570us
I20260709 12:14:55.268383 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.272619 27657 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:55.207202 27409 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.210201 27637 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:14:55.320051 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.316576 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.332625 23395 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:14:55.322181 27671 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:14:55.287729 27585 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:55.313735 27637 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.185559 27579 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:55.364208 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.288937 27532 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.357324 27616 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.266705 27668 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.404639 24014 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:14:55.426199 27677 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:55.408648 27559 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:14:55.390668 27616 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.436429 27668 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.433832 27559 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.408006 27637 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.444175 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:14:55.384481 27549 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:55.445184 27580 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.000s
W20260709 12:14:55.462020 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 193713us
W20260709 12:14:55.462914 23982 leader_election.cc:336] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.839s (SENT)
I20260709 12:14:55.463474 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:55.466377 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 344518us
I20260709 12:14:55.470422 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 851) took 1127 ms. Trace:
I20260709 12:14:55.472425 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.476406 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.477277 23320 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:55.477896 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.474792 27416 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:55.478643 27465 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:55.479110 27465 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.479032 27416 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.479499 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.480171 27465 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.480759 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.476966 26702 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:55.478251 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.470788 23466 rpcz_store.cc:276] 0709 12:14:54.342891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.343028 (+   137us) service_pool.cc:224] Handling call
0709 12:14:54.343305 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:54.492301 (+148996us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:14:54.503508 (+ 11207us) log.cc:844] Serialized 14 byte log entry
0709 12:14:54.967218 (+463710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:54.967299 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:54.967359 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:55.470207 (+502848us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001874d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:55.470250 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:55.470367 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5598,"prepare.queue_time_us":18066,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":45,"raft.queue_time_us":29224,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":6185,"spinlock_wait_cycles":342031232,"thread_start_us":457618,"threads_started":2,"wal-append.queue_time_us":407618}
I20260709 12:14:55.481891 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.487174 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.479063 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.482200 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.483698 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.486507 23313 raft_consensus.cc:2393] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:55.463496 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.495186 27686 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:14:55.496047 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.497097 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.498833 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.500376 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:55.472245 27637 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.503443 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.505600 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.509904 24015 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:14:55.511229 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.506989 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.513602 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.515091 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.474692 27667 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:55.525460 24012 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:14:55.525676 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.525763 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.526518 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.459697 27580 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.477910 27436 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.528024 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.528090 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.533305 24010 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:55.534160 23320 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:55.535686 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.528762 24015 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:14:55.481400 27416 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:55.550835 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.552202 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.553023 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.530725 23982 leader_election.cc:400] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8.
I20260709 12:14:55.554783 23982 leader_election.cc:403] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:55.515410 27554 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:55.529364 27659 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:14:55.580215 27659 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:14:55.565366 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 97114us
I20260709 12:14:55.503747 27568 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:14:55.586217 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.586782 23466 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:55.587827 23464 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:55.529609 27616 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.532861 27532 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.556689 27661 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:55.558318 27657 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:14:55.514894 27686 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:14:55.558466 27665 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.558343 27409 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:14:55.609114 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 78537.6us
I20260709 12:14:55.485594 27559 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.606583 27200 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8.
I20260709 12:14:55.612753 27650 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:14:55.613222 23465 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:55.631525 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:55.614269 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61332.3us
I20260709 12:14:55.638325 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.614719 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.617661 27647 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.618378 27549 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.619241 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:55.620215 27679 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:55.613747 27657 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.650704 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.650715 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.651887 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.651954 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.653440 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.626854 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.654130 23464 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.654474 23389 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:14:55.629233 27586 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.645012 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.641268 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.614120 27640 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:14:55.656738 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:55.625478 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.654830 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.667625 23470 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 10. Ops: []
I20260709 12:14:55.650087 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.624557 27661 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.668323 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.674935 23468 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:14:55.656795 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.661348 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.645462 23467 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.668956 23470 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:55.669374 27666 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:55.669926 23393 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.670710 27693 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:55.671950 27586 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:55.673030 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47658.4us
W20260709 12:14:55.648288 27652 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.688s	user 0.011s	sys 0.002s
I20260709 12:14:55.669071 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.675202 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.638890 27650 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.686952 23467 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:55.680540 23461 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:55.665642 27585 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:55.686100 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.653321 27657 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:55.677822 23394 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:55.687057 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 158769us
I20260709 12:14:55.658401 27598 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:55.682161 23464 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:55.697268 27693 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.678751 27200 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:14:55.681267 27549 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:55.701248 27681 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.702616 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:55.705087 27409 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:14:55.683192 27532 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.716646 27434 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.719714 27532 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.720722 27681 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:55.722677 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.723429 27681 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:55.724153 27683 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:14:55.725943 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:55.725732 27693 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.726630 23396 raft_consensus.cc:2368] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:14:55.730075 27486 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:55.729779 27650 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.730520 27200 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:55.730607 27691 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:55.731626 27200 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.732208 27580 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:55.734761 27586 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:55.731626 27661 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:55.737076 27348 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.000s
I20260709 12:14:55.738744 24008 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:14:55.714488 27549 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.696074 27652 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.743820 27549 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.744957 23390 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:14:55.745105 27549 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:55.749575 27656 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.750887 27656 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:14:55.751812 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.710510 23466 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:55.652945 27647 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.712093 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:55.748000 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44127.2us
I20260709 12:14:55.765892 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:55.766010 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48038us
I20260709 12:14:55.746399 27652 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.763190 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.722085 27532 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.748351 23314 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:55.764577 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.767688 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.768287 27681 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:14:55.784235 23320 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:14:55.773258 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:14:55.777670 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65616us
I20260709 12:14:55.782680 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.787671 23317 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.792567 23315 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:14:55.793480 27558 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:55.794413 27532 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.794673 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.795109 27532 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:55.783215 27554 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:14:55.796115 27532 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.796389 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.796800 27532 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:55.797546 27640 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:55.808643 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.809372 23387 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:55.787705 23982 leader_election.cc:400] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:14:55.803205 27692 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:14:55.798457 27665 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:55.810969 27665 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.811957 27686 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:55.799332 27465 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:55.799913 27659 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:55.800372 27416 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:55.801652 27409 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:55.797166 27532 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.800602 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:55.782935 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:55.804888 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.714707 27598 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:55.807633 23389 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 6. Current term is 7. Ops: []
I20260709 12:14:55.813627 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.829931 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.814299 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:55.830255 27480 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:55.830838 23387 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:55.814744 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:55.815034 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.820815 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:55.832701 23314 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:55.832834 27691 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:14:55.824740 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.833699 23466 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.835023 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:55.836279 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.825381 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.837025 23462 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:55.826550 23393 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:14:55.782061 27662 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:14:55.821164 27612 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:55.844796 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44298.5us
I20260709 12:14:55.826496 23396 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:55.845695 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:55.846726 23466 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:14:55.821625 27668 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:14:55.849086 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:55.850942 24012 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:14:55.850781 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:55.851980 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.864396 27685 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:55.834055 27321 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:55.868574 27585 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:55.869372 27585 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:55.852294 27662 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.870102 27585 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.870679 27662 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.817713 27416 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:55.878046 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:55.878675 23466 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:55.878885 23317 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:55.777247 27670 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:14:55.882248 23461 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:14:55.826195 27665 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:55.826350 27640 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.893743 27556 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:14:55.894001 27480 consensus_queue.cc:1243] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:14:55.894366 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47676.4us
I20260709 12:14:55.894655 27480 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:14:55.896060 24012 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:55.897250 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:55.904282 23469 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:55.813637 27558 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:14:55.818228 27650 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:14:55.908669 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.853672 27661 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:14:55.906213 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81770.7us
I20260709 12:14:55.906595 23461 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:55.875622 26702 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:14:55.828409 27573 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:14:55.908115 27585 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:55.903620 24015 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:14:55.909581 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:55.910841 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:55.918470 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:55.889508 27559 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:14:55.891914 27616 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.889220 27348 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:14:55.894044 27681 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:14:55.930016 23217 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 7 to 9, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:55.928786 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.931445 24015 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:55.874902 27416 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.932340 23463 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.907076 27665 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.939092 27670 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:55.942039 27650 consensus_queue.cc:1243] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:14:55.938457 27616 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.936887 27664 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:14:55.926890 27559 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:55.924192 27685 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:14:55.944332 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:55.953941 24015 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:55.948302 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:55.955984 24016 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:55.948043 23217 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:55.927466 27661 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:55.972481 23217 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 12, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:55.977510 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:55.935915 27681 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:55.949932 27559 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:55.951754 27416 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:55.987790 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 92201.8us
I20260709 12:14:55.949028 27616 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:56.007550 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:55.946857 27650 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:55.994315 23463 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:55.906126 27532 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.974686 27637 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:14:56.008695 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 205559us
I20260709 12:14:56.014274 23217 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 8 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:56.049335 23464 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:14:56.049919 23464 pending_rounds.cc:85] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 7
I20260709 12:14:56.050194 23464 pending_rounds.cc:107] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:14:56.049279 23217 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:56.032675 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.035945 23394 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:14:56.020439 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:56.051465 23465 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.043601 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.032415 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.051819 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:56.022605 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:56.052915 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:56.053203 23470 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:14:56.054314 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:56.054893 23396 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:55.941686 27665 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.992273 27640 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:55.974642 27409 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:14:56.069664 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:56.001173 27692 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:56.028671 24015 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:14:56.072021 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.064406 23464 raft_consensus.cc:2889] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:56.073287 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:55.946614 27670 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:55.943828 27694 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 12:14:55.959748 27598 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.014804 27650 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.006654 27686 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.088s
I20260709 12:14:56.054610 27710 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:56.048219 27637 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.067287 27586 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:14:56.128296 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 55064.9us
I20260709 12:14:56.062943 27654 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:14:56.083360 27692 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:56.067760 27647 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:56.143821 23207 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.293s	user 0.002s	sys 0.007s
I20260709 12:14:56.084211 27616 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:56.167855 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 137344us
W20260709 12:14:56.165838 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89041us
I20260709 12:14:56.168923 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 2 to 6, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 6 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:56.169884 27465 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:14:56.183622 23395 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:14:56.153586 27586 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.170874 27668 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:14:56.172021 23216 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 327) took 25763 ms (client timeout 15000 ms). Trace:
I20260709 12:14:56.154119 27654 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.174721 27665 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:56.154407 27692 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.128821 27480 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:56.175671 23465 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:56.175832 27691 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:56.110872 27532 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:56.182991 23396 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:56.183233 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:56.252638 23463 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:14:56.214951 27697 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.261s	user 0.001s	sys 0.000s
I20260709 12:14:56.229400 23462 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 5.
I20260709 12:14:56.167500 27612 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.237876 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.257740 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:56.268859 23394 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:14:56.239302 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 70815.1us
I20260709 12:14:56.181533 27647 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:56.170904 27321 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:56.181749 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 115581us
I20260709 12:14:56.083782 27681 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.256469 23217 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 8, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:56.227841 27585 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.128187 27661 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.282691 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.233376 27586 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:56.246887 27691 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:56.338040 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.340008 27657 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.582s	user 0.008s	sys 0.001s
I20260709 12:14:56.340473 27657 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.232846 27650 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.342310 27650 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.349828 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.350435 23387 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:14:56.348063 23217 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 6, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 6 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:56.293767 27465 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:56.255235 27480 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.333956 27647 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.371948 23387 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:56.372634 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:56.397054 27723 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:56.402107 23394 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:14:56.332556 27585 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.293247 27612 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.357228 27321 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:14:56.428205 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.453281 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:56.455380 23394 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:56.459599 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.391499 27554 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:14:56.228581 23216 rpcz_store.cc:269] 0709 12:14:30.346237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.346516 (+   279us) service_pool.cc:224] Handling call
0709 12:14:56.104610 (+25758094us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:47.638271 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:47.943288 (+305017us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:47.953958 (+ 10670us) write_op.cc:435] Acquired schema lock
0709 12:14:47.964129 (+ 10171us) tablet.cc:662] Decoding operations
0709 12:14:49.258185 (+1294056us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:49.258280 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:49.258323 (+    43us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:14:49.261038 (+  2715us) tablet.cc:701] Row locks acquired
0709 12:14:49.261062 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:14:49.261284 (+   222us) write_op.cc:270] Start()
0709 12:14:49.261424 (+   140us) write_op.cc:276] Timestamp: P: 1783599289261214 usec, L: 0
0709 12:14:49.280348 (+ 18924us) op_driver.cc:348] REPLICATION: starting
0709 12:14:49.513461 (+233113us) log.cc:844] Serialized 9919 byte log entry
0709 12:14:50.387978 (+874517us) op_driver.cc:464] REPLICATION: finished
0709 12:14:50.723096 (+335118us) write_op.cc:301] APPLY: starting
0709 12:14:50.723221 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:50.874222 (+151001us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:50.874475 (+   253us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:50.907796 (+ 33321us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:50.908389 (+   593us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=50
0709 12:14:50.908494 (+   105us) write_op.cc:312] APPLY: finished
0709 12:14:50.909329 (+   835us) log.cc:844] Serialized 170 byte log entry
0709 12:14:50.947261 (+ 37932us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:50.966624 (+ 19363us) write_op.cc:454] Released schema lock
0709 12:14:50.966773 (+   149us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:50.967736 (+   963us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:50.967789 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2422229,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":316701,"apply.run_cpu_time_us":15303,"apply.run_wall_time_us":252727,"num_ops":25,"prepare.queue_time_us":409306,"prepare.run_cpu_time_us":18715,"prepare.run_wall_time_us":2214231,"raft.queue_time_us":167720,"raft.run_cpu_time_us":56643,"raft.run_wall_time_us":394892,"replication_time_us":1056219,"thread_start_us":835597,"threads_started":4,"wal-append.queue_time_us":289844,"wal-append.run_cpu_time_us":34062,"wal-append.run_wall_time_us":2102654}]]}
I20260709 12:14:56.464247 23396 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:14:56.464970 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.332231 27637 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.468753 23217 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 10, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:14:56.473804 23217 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:56.474541 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:56.475277 23318 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:56.477967 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:56.482126 23465 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:56.484922 23317 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:14:56.485291 23317 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.348129 27654 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.487986 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:56.488201 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.486308 23317 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.497855 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:56.455452 27710 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.466838 27554 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:14:56.519775 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 243851us
W20260709 12:14:56.506963 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 224331us
I20260709 12:14:56.522375 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.504387 27486 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:56.509451 23394 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:14:56.480343 26702 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:56.557025 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 79135.3us
I20260709 12:14:56.556834 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.503545 27579 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:14:56.573159 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.499594 27670 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:56.470460 27321 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.584139 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:56.593521 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:56.394292 27691 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.556604 27654 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:14:56.602442 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:56.557169 27667 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.578608 26702 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.604990 27707 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:14:56.343255 27708 consensus_queue.cc:1175] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 9 status { last_received { term: 8 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:56.598156 27670 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:56.621207 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:56.624840 27700 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:56.629601 27705 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:14:56.629688 27732 raft_consensus.cc:2764] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:14:56.630023 27480 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:56.618276 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:56.587769 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.614826 27722 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:56.664664 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77023.9us
W20260709 12:14:56.671722 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53525.7us
I20260709 12:14:56.673012 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:56.622718 27715 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:14:56.677973 23314 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:14:56.545608 27647 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.698184 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:14:56.677521 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:14:56.678574 23314 pending_rounds.cc:85] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:14:56.702972 23314 pending_rounds.cc:107] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:14:56.703289 23314 raft_consensus.cc:2889] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:56.518338 27465 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.704948 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.707065 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:56.710990 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.711877 23395 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
W20260709 12:14:56.710191 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.713755 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.713918 23390 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:14:56.661845 27670 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:56.720256 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:56.672578 27722 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.734102 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:56.744658 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.699631 27480 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:14:56.640457 27692 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.723337 27707 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:14:56.749257 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:56.725443 27670 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:56.720196 26702 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:56.725531 27732 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.632418 27667 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.735136 27723 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:14:56.738060 27722 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:14:56.769315 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62326.6us
I20260709 12:14:56.665450 27554 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.752714 27480 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:56.754045 27657 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:56.766861 27704 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 12:14:56.671357 27700 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.671952 27710 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.770550 27732 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.785593 27585 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:56.802803 27598 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 12:14:56.794588 27416 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.781s	user 0.010s	sys 0.007s
I20260709 12:14:56.784809 27612 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:56.815832 23394 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:56.788364 27480 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:56.793807 27486 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:14:56.820737 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:56.834100 27699 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 12:14:56.836615 27692 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:56.837368 27750 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:56.837755 27321 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:56.838018 27750 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:56.840670 23312 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:14:56.820446 27554 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:56.813222 27707 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:14:56.833269 27656 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.082s	user 0.013s	sys 0.000s
I20260709 12:14:56.830574 27720 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:56.874123 27720 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.874635 27720 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:14:56.877509 27745 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:14:56.881459 27586 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 12:14:56.882027 27586 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:56.869331 27667 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:56.869935 27737 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:14:56.871639 27654 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:56.851344 27409 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:56.832612 27585 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.100359 23464 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:14:57.002012 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:57.075562 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:57.031798 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.113111 23465 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
W20260709 12:14:56.859951 27734 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:14:56.837409 27650 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:14:57.007835 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:57.127370 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:56.982963 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 462294us
I20260709 12:14:57.074959 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:56.929011 27532 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
W20260709 12:14:56.903631 27559 consensus_meta.cc:287] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.890s	user 0.008s	sys 0.003s
I20260709 12:14:57.074783 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:56.845721 27486 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:14:56.830472 27727 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:14:56.982586 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 252884us
I20260709 12:14:57.125810 23390 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:14:57.152976 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 78104.5us
I20260709 12:14:57.153851 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:14:57.154685 23467 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:14:57.155774 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.156047 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.125679 27754 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.156980 23316 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:57.158672 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.158773 23314 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:14:57.160216 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:56.898666 27693 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:14:56.897593 23315 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:14:57.110599 27707 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.148406 23207 maintenance_manager.cc:643] P 259b912070e34dfba6c09de6bbb3d141: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.913s	user 0.179s	sys 0.115s Metrics: {"bytes_written":30633,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":370834,"dirs.run_cpu_time_us":641,"dirs.run_wall_time_us":99328,"drs_written":1,"lbm_read_time_us":39765,"lbm_reads_1-10_ms":2,"lbm_reads_10-100_ms":2,"lbm_write_time_us":421949,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":10,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":285901,"threads_started":2,"update_count":25,"wal-append.queue_time_us":170811}
I20260709 12:14:57.127077 27650 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:56.859895 27732 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.160929 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:56.841115 27753 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:14:57.163940 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.123600 27409 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.162856 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:57.104712 27750 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:57.112779 27700 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:57.028317 27725 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.431s	user 0.000s	sys 0.003s
W20260709 12:14:57.202028 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48242.2us
I20260709 12:14:57.007118 27720 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:57.264671 27748 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.275s	user 0.003s	sys 0.000s
I20260709 12:14:57.230242 23464 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:57.246049 27656 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.275422 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2210) took 1218 ms. Trace:
I20260709 12:14:57.158035 27755 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:14:57.225353 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 78549.2us
I20260709 12:14:56.850499 27723 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.261907 24015 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:14:57.223140 23208 maintenance_manager.cc:419] P 259b912070e34dfba6c09de6bbb3d141: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10634 bytes on disk
I20260709 12:14:57.271812 27666 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:57.266741 27650 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:14:57.166412 27693 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:14:57.345548 27558 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.246016 27737 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.255403 24008 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:14:57.228580 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:57.165678 27532 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:57.123023 27585 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.330605 27348 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.257458 27727 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.231284 23468 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:14:57.230456 27747 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.229647 27486 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.232177 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:57.421797 23469 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:14:57.256989 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:57.285732 27637 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:57.195878 27757 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.288455 27694 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:57.287668 23317 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:14:57.426987 23207 maintenance_manager.cc:643] P 259b912070e34dfba6c09de6bbb3d141: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.036s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":1376,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3}
I20260709 12:14:57.164927 27647 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 12:14:57.121336 27434 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.327623 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:57.171286 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:57.357457 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:57.479197 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 307997us
W20260709 12:14:57.479449 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 122128us
I20260709 12:14:57.127192 27654 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.368461 23467 rpcz_store.cc:276] 0709 12:14:56.057348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.097040 (+ 39692us) service_pool.cc:224] Handling call
0709 12:14:56.097491 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.155068 (+1057577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.275136 (+120068us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:57.275352 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2430817024}
I20260709 12:14:57.369867 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:14:57.331166 27670 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.006s
I20260709 12:14:57.165215 27549 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:14:57.346555 27321 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:57.167896 27754 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.360351 27753 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:14:57.165421 27579 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.001s
I20260709 12:14:57.333395 27686 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.163307 27559 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.511184 27729 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:57.391695 27558 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:57.481104 23287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.640s (SENT) [suppressed 22 similar messages]
I20260709 12:14:57.481525 27710 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.487186 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.392298 27556 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:14:57.376631 27656 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.420513 27401 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.388381 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:14:57.547358 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 159052us
I20260709 12:14:57.426949 27715 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.506057 23317 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:14:57.548185 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:57.550622 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2322) took 1076 ms. Trace:
I20260709 12:14:57.551183 23317 rpcz_store.cc:276] 0709 12:14:56.473835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.473971 (+   136us) service_pool.cc:224] Handling call
0709 12:14:56.484779 (+ 10808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.287076 (+802297us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b58007037d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4
0709 12:14:57.550322 (+263246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.550564 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38737152}
I20260709 12:14:57.551752 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:57.552477 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:57.508572 23465 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:14:57.552949 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:57.495908 27434 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.565569 23392 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:57.486693 27667 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.504171 27670 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.480759 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:14:57.537927 27579 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:14:57.422361 27348 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.406423 27700 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.530546 23387 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:14:57.530509 23313 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:14:57.447850 27727 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.392813 27737 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:57.529399 27480 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.002s
I20260709 12:14:57.559566 27568 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:14:57.406049 27532 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:57.572044 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 91828.6us
I20260709 12:14:57.486084 27747 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.487740 27554 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:14:57.585175 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 104089us
I20260709 12:14:57.496198 27757 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.490126 27694 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.586138 27579 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:57.591972 23388 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:57.592691 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:57.571189 27436 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:14:57.601303 27700 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.601395 27741 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 12:14:57.603492 27200 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:57.605667 27752 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:57.605592 27753 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.648643 24012 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:14:57.649118 24012 pending_rounds.cc:85] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 8
I20260709 12:14:57.649348 24012 pending_rounds.cc:107] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:14:57.649601 24012 raft_consensus.cc:2889] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:57.573132 27715 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.510540 27744 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:57.596946 27552 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:57.628961 27568 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:14:57.501209 27693 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:57.610046 23320 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:14:57.582197 27758 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.571312 27555 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:14:57.559928 27465 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.585597 27586 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.588864 27661 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.619016 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:57.630745 24014 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:57.634963 27701 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.128s
W20260709 12:14:57.612032 27722 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.004s
I20260709 12:14:57.590493 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:57.527063 27686 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.561674 27401 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.652674 24008 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:14:57.530093 27729 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.585920 27667 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.651518 23390 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:14:57.571786 27718 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:57.657407 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 176706us
I20260709 12:14:57.643625 27710 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.644727 27579 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.614698 27480 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.653831 27769 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.648437 27741 consensus_queue.cc:1243] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:14:57.677287 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58347.8us
I20260709 12:14:57.642644 27554 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.656594 27752 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.650513 27573 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:14:57.656433 27200 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.664871 27409 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.676084 27693 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.714896 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:14:57.716256 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:57.717736 23315 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.006s
I20260709 12:14:57.685231 27549 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:57.723839 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:14:57.677817 27585 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.730549 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:14:57.747886 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56194.5us
I20260709 12:14:57.688047 27758 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.752224 24012 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:14:57.703411 27767 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:58.123015 27758 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:58.123641 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:14:57.708431 27768 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:14:58.124972 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1868) took 1123 ms. Trace:
I20260709 12:14:58.125409 27758 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:58.125322 23396 rpcz_store.cc:276] 0709 12:14:57.001534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.001654 (+   120us) service_pool.cc:224] Handling call
0709 12:14:57.002032 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.119824 (+117792us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:14:57.120128 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:14:57.421101 (+300973us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.465776 (+ 44675us) log.cc:844] Serialized 26 byte log entry
0709 12:14:57.465981 (+   205us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:57.466031 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.466127 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.124778 (+658651us) raft_consensus.cc:1685] finished
0709 12:14:58.124797 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.124914 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199718,"prepare.queue_time_us":608,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":143737,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":66022,"spinlock_wait_cycles":35456,"thread_start_us":300572,"threads_started":1,"wal-append.queue_time_us":461650}
I20260709 12:14:58.126997 27742 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:14:57.893978 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 177814us
I20260709 12:14:57.702523 27754 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:57.709877 27757 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.219187 27757 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:58.219760 27757 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:58.222024 27757 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.707587 27772 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:14:57.711831 27737 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.792850 27747 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.800369 27759 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:14:57.661744 27744 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.702020 27771 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.160s
I20260709 12:14:57.804858 27753 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:14:57.831403 27777 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:58.248293 27777 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.840880 27775 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:58.249693 27775 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:58.251883 27775 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:57.841903 27780 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:14:57.852383 27752 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:14:57.862290 27783 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:14:57.726552 27769 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:58.256579 27769 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.954665 27784 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:58.257373 27784 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:58.259677 27777 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.955030 27774 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:58.003333 23320 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:14:58.024731 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:58.261546 23390 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
W20260709 12:14:58.026396 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.050186 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2499) took 1341 ms. Trace:
W20260709 12:14:58.052426 27751 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.001s	sys 0.009s
W20260709 12:14:58.072695 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2486) took 2423 ms (client timeout 1640 ms). Trace:
I20260709 12:14:58.090680 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:58.095005 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 740) took 2062 ms. Trace:
I20260709 12:14:58.100219 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:14:58.104123 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:58.107563 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:58.109419 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:58.117093 27794 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:14:58.118162 27793 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:58.127717 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:57.916298 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110691us
I20260709 12:14:58.105034 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:14:58.555704 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 450727us
I20260709 12:14:58.556162 27552 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:14:58.144526 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:14:57.801522 24008 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:14:58.147977 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2320) took 1674 ms. Trace:
I20260709 12:14:58.296581 23313 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:58.890708 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:58.891904 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:58.271102 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:14:59.853821 27810 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.854276 23320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2342) took 1765 ms (client timeout 1964 ms). Trace:
W20260709 12:14:58.201555 23315 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2328) took 1606 ms (client timeout 1703 ms). Trace:
I20260709 12:14:58.869211 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1074) took 1364 ms. Trace:
I20260709 12:14:58.271337 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:14:58.870007 23312 rpcz_store.cc:276] 0709 12:14:56.473086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.475641 (+  2555us) service_pool.cc:224] Handling call
0709 12:14:56.475885 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:56.841016 (+365131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.147711 (+1306695us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54003c0d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:58.147906 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838136576}
I20260709 12:14:58.182647 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2327) took 1663 ms. Trace:
I20260709 12:14:59.930508 23316 rpcz_store.cc:276] 0709 12:14:56.519229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.520119 (+   890us) service_pool.cc:224] Handling call
0709 12:14:56.520414 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.157318 (+636904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.182352 (+1025034us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54001a7cd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:58.182574 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463246208}
I20260709 12:14:59.932102 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:59.933048 23316 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:58.194135 23317 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:14:59.936995 23317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1076) took 2418 ms (client timeout 3205 ms). Trace:
I20260709 12:14:58.199694 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:59.939765 23318 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:14:58.202116 23321 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:14:58.263803 23387 rpcz_store.cc:276] 0709 12:14:56.709157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.710670 (+  1513us) service_pool.cc:224] Handling call
0709 12:14:56.711136 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.598471 (+887335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.049844 (+451373us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54002dfd54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:58.050074 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884131200}
I20260709 12:14:59.958618 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.959306 23387 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
W20260709 12:14:59.959793 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2511) took 1667 ms (client timeout 1802 ms). Trace:
W20260709 12:14:59.960139 23387 rpcz_store.cc:269] 0709 12:14:58.292617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.958312 (+1665695us) service_pool.cc:224] Handling call
0709 12:14:59.959725 (+  1413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:58.297904 23388 raft_consensus.cc:2410] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:14:58.293529 23389 rpcz_store.cc:276] 0709 12:14:56.032667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.032799 (+   132us) service_pool.cc:224] Handling call
0709 12:14:56.033213 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:56.043553 (+ 10340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:56.043579 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:56.095193 (+ 51614us) log.cc:844] Serialized 52 byte log entry
0709 12:14:56.492870 (+397677us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:14:56.724143 (+231273us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.749715 (+ 25572us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.749898 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.749976 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:57.046885 (+296909us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b180033fd00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1
0709 12:14:57.150392 (+103507us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b180033fd00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1
0709 12:14:57.259578 (+109186us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b180033fd00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:57.632941 (+373363us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b180033fd00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:57.884195 (+251254us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b180033fd00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:14:58.094783 (+210588us) raft_consensus.cc:1685] finished
0709 12:14:58.094804 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.094950 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35069,"prepare.queue_time_us":101512,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":2650,"raft.queue_time_us":95131,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":33158,"spinlock_wait_cycles":55161984,"thread_start_us":520458,"threads_started":2,"wal-append.queue_time_us":656136}
I20260709 12:14:59.964103 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.964743 23389 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:14:58.292140 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:58.296967 23392 rpcz_store.cc:269] 0709 12:14:55.648815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.652761 (+  3946us) service_pool.cc:224] Handling call
0709 12:14:58.072486 (+2419725us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400190ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:58.072616 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4212429184}
I20260709 12:14:59.968866 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.969681 23392 raft_consensus.cc:2410] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:14:59.970058 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 744) took 1857 ms (client timeout 2393 ms). Trace:
W20260709 12:14:59.970386 23392 rpcz_store.cc:269] 0709 12:14:58.112633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.968603 (+1855970us) service_pool.cc:224] Handling call
0709 12:14:59.970008 (+  1405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.971200 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.971868 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2512) took 1678 ms. Trace:
I20260709 12:14:59.972152 23392 rpcz_store.cc:276] 0709 12:14:58.293032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.970974 (+1677942us) service_pool.cc:224] Handling call
0709 12:14:59.971818 (+   844us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.972951 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:58.299942 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.974313 23396 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:14:59.974815 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 753) took 1859 ms (client timeout 1977 ms). Trace:
W20260709 12:14:59.975158 23396 rpcz_store.cc:269] 0709 12:14:58.115138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.296378 (+181240us) service_pool.cc:224] Handling call
0709 12:14:59.974745 (+1678367us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:58.557176 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 413760us
I20260709 12:14:58.219187 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:14:58.316742 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 191370us
I20260709 12:14:59.983142 27815 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:14:58.490139 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2207) took 2596 ms. Trace:
I20260709 12:14:59.976120 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:14:59.987418 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 747) took 1874 ms. Trace:
I20260709 12:14:59.986243 23466 rpcz_store.cc:276] 0709 12:14:55.893845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:55.896682 (+  2837us) service_pool.cc:224] Handling call
0709 12:14:55.896978 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:55.897325 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:55.897354 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:55.897973 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 12:14:56.368372 (+470399us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:56.501716 (+133344us) log.cc:844] Serialized 14 byte log entry
0709 12:14:56.699834 (+198118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:56.699934 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:57.027519 (+327585us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:14:58.141468 (+1113949us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b24002ca068. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a640d9c4 00007ff0a640bacb 00007ff0a65c6dc9 00007ff0a65c6a55 00007ff0a65c5fb0 00007ff0a923c488 00007ff0a923cbe8 00007ff0a9214398 00007ff0a920f76a 00007ff0a920ebb6
0709 12:14:58.489841 (+348373us) raft_consensus.cc:1685] finished
0709 12:14:58.489868 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.490064 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6006,"prepare.queue_time_us":310634,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":88077,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":15815,"spinlock_wait_cycles":41076352,"thread_start_us":744168,"threads_started":3,"wal-append.queue_time_us":579221,"wal-append.run_cpu_time_us":22994,"wal-append.run_wall_time_us":2691520}
I20260709 12:14:59.987751 23396 rpcz_store.cc:276] 0709 12:14:58.113308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.975841 (+1862533us) service_pool.cc:224] Handling call
0709 12:14:59.987342 (+ 11501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:58.517524 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:58.573705 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1869) took 1460 ms. Trace:
I20260709 12:14:58.419193 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:59.991101 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:14:59.991729 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 878) took 1572 ms. Trace:
I20260709 12:14:59.991961 23466 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:14:59.992110 23462 rpcz_store.cc:276] 0709 12:14:58.418819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.418933 (+   114us) service_pool.cc:224] Handling call
0709 12:14:59.991671 (+1572738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:58.440711 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:59.994315 23464 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 876) took 1576 ms (client timeout 1868 ms). Trace:
W20260709 12:14:59.994666 23464 rpcz_store.cc:269] 0709 12:14:58.418240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.440338 (+ 22098us) service_pool.cc:224] Handling call
0709 12:14:59.994257 (+1553919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.995492 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:58.430068 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:59.996906 23467 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 875) took 1579 ms (client timeout 2043 ms). Trace:
W20260709 12:14:59.997254 23467 rpcz_store.cc:269] 0709 12:14:58.417963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.425394 (+  7431us) service_pool.cc:224] Handling call
0709 12:14:59.996849 (+1571455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.998275 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:14:59.998988 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2221) took 1207 ms. Trace:
I20260709 12:14:59.999312 23467 rpcz_store.cc:276] 0709 12:14:58.791906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.997930 (+1206024us) service_pool.cc:224] Handling call
0709 12:14:59.998934 (+  1004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.998831 27815 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:14:58.502766 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.001166 23469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 881) took 1500 ms (client timeout 1863 ms). Trace:
W20260709 12:15:00.001509 23469 rpcz_store.cc:269] 0709 12:14:58.501119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.502538 (+  1419us) service_pool.cc:224] Handling call
0709 12:15:00.001112 (+1498574us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.002863 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 884) took 1497 ms. Trace:
I20260709 12:15:00.003192 23469 rpcz_store.cc:276] 0709 12:14:58.505560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.002086 (+1496526us) service_pool.cc:224] Handling call
0709 12:15:00.002355 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.002547 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.002575 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.002619 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.002652 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.002705 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.002802 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:58.524623 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:58.866400 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.006590 23466 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:14:59.613128 27803 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.007289 27803 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:59.990939 23393 rpcz_store.cc:276] 0709 12:14:57.113718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.115009 (+  1291us) service_pool.cc:224] Handling call
0709 12:14:57.115291 (+   282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:57.370824 (+255533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.370853 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:57.371494 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 12:14:58.191449 (+819955us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18002a2fe0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:14:58.295036 (+103587us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:14:58.295091 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.573319 (+278228us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b5400344ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:58.573376 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.573461 (+    85us) raft_consensus.cc:1685] finished
0709 12:14:58.573477 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.573639 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1408,"raft.queue_time_us":4733,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":8909,"spinlock_wait_cycles":590591104,"thread_start_us":103071,"threads_started":1,"wal-append.queue_time_us":915297,"wal-append.run_cpu_time_us":7253,"wal-append.run_wall_time_us":1002175}
I20260709 12:14:59.793200 27804 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:14:59.801805 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002ab400 after lost signal to thread 22110
I20260709 12:14:59.818899 27806 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.824637 27805 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:00.010387 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800330840 after lost signal to thread 23253
I20260709 12:14:57.782835 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:57.787281 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:58.132324 24012 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:00.050858 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1021) took 1919 ms. Trace:
I20260709 12:15:00.051220 24012 rpcz_store.cc:276] 0709 12:14:58.131596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.131749 (+   153us) service_pool.cc:224] Handling call
0709 12:14:58.132061 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.050556 (+1918495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.050791 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.052562 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:57.812785 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:58.526564 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:58.134795 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:58.343169 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 744) took 1144 ms. Trace:
I20260709 12:14:57.906683 27321 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:14:57.989049 27434 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.878165 27554 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.755219 27559 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.752002 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:14:57.757090 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.056815 27821 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:00.064554 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 980) took 2339 ms (client timeout 1827 ms). Trace:
I20260709 12:15:00.064503 27821 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:00.065101 24009 rpcz_store.cc:269] 0709 12:14:57.725645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.725766 (+   121us) service_pool.cc:224] Handling call
0709 12:15:00.064480 (+2338714us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.066375 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.066911 27821 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:00.067255 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 750) took 1826 ms (client timeout 1963 ms). Trace:
W20260709 12:15:00.067752 24009 rpcz_store.cc:269] 0709 12:14:58.241077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.066103 (+1825026us) service_pool.cc:224] Handling call
0709 12:15:00.067187 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:58.004563 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:14:58.442288 27647 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.955950 27666 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:57.956429 27580 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.057114 24014 rpcz_store.cc:276] 0709 12:14:57.198714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.330433 (+131719us) service_pool.cc:224] Handling call
0709 12:14:58.342973 (+1012540us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540074cf54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:58.343086 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299898112}
I20260709 12:15:00.070873 27666 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.071426 27580 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.072132 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.072928 24014 raft_consensus.cc:2410] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:14:58.353286 27348 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:57.849906 27480 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.955507 27586 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.078456 27586 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.081068 27586 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.705142 27656 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.083370 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 877) took 1664 ms. Trace:
I20260709 12:15:00.083724 23461 rpcz_store.cc:276] 0709 12:14:58.418531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.431062 (+ 12531us) service_pool.cc:224] Handling call
0709 12:14:58.431406 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.082535 (+1651129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.082566 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.082608 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.082638 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.083145 (+   507us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54002256d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.083185 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.083309 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3796792704}
I20260709 12:14:58.115870 27662 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:14:57.797327 27200 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.698767 27401 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.678136 27465 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.091153 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 887) took 1582 ms. Trace:
I20260709 12:14:57.827386 27532 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.091938 27465 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.556644 27552 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.094214 27552 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:14:57.746971 27556 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.098279 27556 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.794328 27558 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.696606 27568 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.878667 27579 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.118062 27579 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.095924 27625 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.119395 27625 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.120709 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 743) took 2008 ms. Trace:
I20260709 12:14:57.883172 27650 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.122831 23319 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:00.121081 23393 rpcz_store.cc:276] 0709 12:14:58.112376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.008735 (+1896359us) service_pool.cc:224] Handling call
0709 12:15:00.009079 (+   344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:00.044004 (+ 34925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.044032 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:00.044762 (+   730us) log.cc:844] Serialized 52 byte log entry
0709 12:15:00.092247 (+ 47485us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.104870 (+ 12623us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.120187 (+ 15317us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.120380 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.120452 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:00.120508 (+    56us) raft_consensus.cc:1685] finished
0709 12:15:00.120521 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.120650 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13700,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":36056,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":220,"thread_start_us":93722,"threads_started":3,"wal-append.queue_time_us":71057}
I20260709 12:15:00.123616 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:00.123752 27650 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.123915 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1767) took 2965 ms. Trace:
I20260709 12:15:00.124168 23393 raft_consensus.cc:606] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:14:58.040019 27652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.001s
W20260709 12:15:00.127235 27652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.627s	user 0.009s	sys 0.002s
I20260709 12:14:57.689271 27654 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.708155 27659 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.129748 27659 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:57.679530 27661 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.134274 27661 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:57.799919 27667 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.392443 27668 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:58.076556 27670 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.139264 27668 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:58.334118 27685 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.850878 27686 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.140786 27685 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:59.828558 27807 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.141543 27668 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.065981 27691 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:14:57.791509 27694 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.392032 27701 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.141741 27807 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.146865 27701 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.148401 27807 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.148941 27701 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.338609 27707 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.150303 27707 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.152323 27707 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.726827 27710 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.154906 27710 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:57.699265 27715 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.027493 27716 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.726975 27718 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.157832 27716 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.159901 27716 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.787268 27719 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:00.161134 27719 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.163354 27719 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.364634 27720 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.119451 27721 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:14:57.685837 27722 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:58.449062 27723 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:57.868101 27727 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.167184 27723 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:59.195960 27732 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.168542 27732 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.169227 27723 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.170745 27732 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.732075 27737 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.171927 27737 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:58.225697 27744 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
W20260709 12:14:58.478626 27751 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.003s	sys 0.012s
I20260709 12:14:59.213133 27755 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:14:57.706482 27700 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:58.014815 27708 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:14:58.795284 27747 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:14:58.733844 27759 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.182371 27759 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:14:58.182909 27764 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.632s	user 0.003s	sys 0.000s
I20260709 12:14:58.793552 27753 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.209506 27757 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.190284 27757 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.192605 27757 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.868124 27769 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:00.193895 27769 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.196219 27769 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.860643 27770 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.197724 23390 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:15:00.200229 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 751) took 2085 ms (client timeout 1868 ms). Trace:
W20260709 12:15:00.200639 23390 rpcz_store.cc:269] 0709 12:14:58.114681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.291769 (+177088us) service_pool.cc:224] Handling call
0709 12:15:00.200029 (+1908260us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b5400190854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.200158 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":528817024}
I20260709 12:14:58.794697 27771 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:00.202256 27771 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.204671 27771 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.952179 27775 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.206367 27775 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.208465 27775 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:14:58.483435 27777 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.210178 27777 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.212911 27777 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:14:59.030088 27780 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.215257 27780 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:14:58.557850 27782 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.218395 27782 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.218827 27782 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:14:58.301107 27793 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:14:58.305830 27794 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:14:59.832376 27809 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.225464 27809 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.227950 27809 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:14:59.833638 22112 debug-util.cc:398] Leaking SignalData structure 0x7b08003068e0 after lost signal to thread 23287
I20260709 12:14:58.566212 27785 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:14:58.259938 27784 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:14:58.384629 27774 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:59.854638 27810 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.242975 27810 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:14:59.856036 23315 rpcz_store.cc:269] 0709 12:14:56.595236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:56.595375 (+   139us) service_pool.cc:224] Handling call
0709 12:14:58.201328 (+1605953us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:58.201494 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550152960}
I20260709 12:15:00.244349 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:14:59.854660 23320 rpcz_store.cc:269] 0709 12:14:58.088912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.270210 (+181298us) service_pool.cc:224] Handling call
0709 12:14:59.854210 (+1584000us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.877923 27811 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:14:59.900902 27812 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:59.905754 23314 rpcz_store.cc:276] 0709 12:14:57.504748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.505042 (+   294us) service_pool.cc:224] Handling call
0709 12:14:57.505412 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:57.652367 (+146955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.652397 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:57.652435 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:14:57.652474 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:58.868982 (+1216508us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54003f9dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:58.869028 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:58.869158 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337515392}
I20260709 12:14:59.942386 27808 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:59.941583 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 1.67025e+06us
I20260709 12:14:59.943058 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1069) took 2517 ms. Trace:
W20260709 12:14:59.940420 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2345) took 1838 ms (client timeout 1945 ms). Trace:
I20260709 12:15:00.246672 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:00.246831 23318 rpcz_store.cc:269] 0709 12:14:58.102343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.199425 (+ 97082us) service_pool.cc:224] Handling call
0709 12:14:59.940341 (+1740916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:59.946791 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:00.247430 23314 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:14:59.937368 23317 rpcz_store.cc:269] 0709 12:14:57.519176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.552210 (+ 33034us) service_pool.cc:224] Handling call
0709 12:14:59.936813 (+2384603us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b5400388454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:59.936936 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1474213632}
I20260709 12:14:59.947760 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1089) took 1298 ms. Trace:
I20260709 12:14:59.945890 27814 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:14:59.963349 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2506) took 2233 ms. Trace:
I20260709 12:15:00.245791 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:14:59.961141 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:14:59.973542 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:14:59.984356 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46548.5us
I20260709 12:15:00.249086 23388 rpcz_store.cc:276] 0709 12:14:57.729799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.730629 (+   830us) service_pool.cc:224] Handling call
0709 12:14:59.963285 (+2232656us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.249716 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 754) took 2134 ms. Trace:
I20260709 12:14:59.988667 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:14:59.990465 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 886) took 1484 ms (client timeout 1906 ms). Trace:
I20260709 12:14:58.497219 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:14:59.271996 23417 debug-util.cc:398] Leaking SignalData structure 0x7b0800162d00 after lost signal to thread 22110
I20260709 12:15:00.010063 27806 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:14:59.469635 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53686.2us
I20260709 12:15:00.262099 27806 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.185334 27759 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.264032 27744 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.264568 27744 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:14:59.993155 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.265865 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2224) took 1473 ms. Trace:
I20260709 12:15:00.011065 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2225) took 1218 ms. Trace:
I20260709 12:15:00.266191 23462 rpcz_store.cc:276] 0709 12:14:58.792711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.992928 (+1200217us) service_pool.cc:224] Handling call
0709 12:15:00.265802 (+272874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.266418 23466 rpcz_store.cc:276] 0709 12:14:58.792956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.990843 (+1197887us) service_pool.cc:224] Handling call
0709 12:15:00.010976 (+ 20133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.267225 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.267516 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.267728 23462 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:00.268314 23466 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:00.268108 27782 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.268695 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.011543 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08000e59a0 after lost signal to thread 23345
I20260709 12:15:00.026821 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:00.038453 27820 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:00.009263 27804 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.010550 27805 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.271528 27820 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.272428 27806 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:00.273020 27806 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:00.041668 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 1.82245e+06us
I20260709 12:15:00.039472 27803 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.043427 24007 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 745) took 2280 ms (client timeout 2000 ms). Trace:
I20260709 12:15:00.047183 27813 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.047806 27817 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:00.048635 23218 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 6 to 9, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 9 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:00.278666 24013 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:00.055344 24015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 748) took 1814 ms (client timeout 1562 ms). Trace:
I20260709 12:15:00.055744 24016 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:00.053333 24012 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 752) took 1783 ms (client timeout 1786 ms). Trace:
I20260709 12:15:00.059005 27321 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.280426 27820 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.063465 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 317) took 18699 ms (client timeout 15000 ms). Trace:
W20260709 12:15:00.069700 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 984) took 2269 ms (client timeout 1787 ms). Trace:
I20260709 12:15:00.063496 24008 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.070286 27647 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.073966 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:00.281493 24010 rpcz_store.cc:269] 0709 12:14:57.800171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.004302 (+204131us) service_pool.cc:224] Handling call
0709 12:15:00.069624 (+2065322us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.279215 24015 rpcz_store.cc:269] 0709 12:14:58.240582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.526285 (+285703us) service_pool.cc:224] Handling call
0709 12:15:00.055266 (+1528981us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.282316 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 747) took 2042 ms (client timeout 2154 ms). Trace:
W20260709 12:15:00.282611 24009 rpcz_store.cc:269] 0709 12:14:58.240310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.073716 (+1833406us) service_pool.cc:224] Handling call
0709 12:15:00.282263 (+208547us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.073285 24014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1023) took 1938 ms (client timeout 2157 ms). Trace:
I20260709 12:15:00.074640 27580 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.073371 27666 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.073920 23464 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:00.076242 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 883) took 1571 ms. Trace:
I20260709 12:14:57.746424 27549 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.091943 23461 rpcz_store.cc:276] 0709 12:14:58.509103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.085113 (+1576010us) service_pool.cc:224] Handling call
0709 12:15:00.090545 (+  5432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.090808 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.090843 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.090899 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.090934 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.090991 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.091097 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.300009 27549 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.300585 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.105190 23436 connection.cc:256] RPC call timeout handler was delayed by -1.3384929010001088s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 12:15:00.105278 27586 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.301779 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.301640 27586 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.302536 23461 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:00.303404 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:00.105887 27556 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.115180 27552 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.301671 27810 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.308766 27810 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.309903 27552 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.124207 23319 rpcz_store.cc:276] 0709 12:14:57.158883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.164550 (+  5667us) service_pool.cc:224] Handling call
0709 12:14:57.164822 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.123248 (+2958426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.123750 (+   502us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b54004114d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.123873 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6802391936}
I20260709 12:15:00.310588 27821 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:00.128846 27625 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.311024 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.311700 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:00.121675 23395 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:15:00.295883 23218 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:14:57.946707 27681 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:00.143328 27823 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.155546 27637 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.160313 27718 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.161137 23391 raft_consensus.cc:2376] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12. Re-sending same reply.
I20260709 12:15:00.166795 23313 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:14:57.702165 27729 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.167732 27722 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.174592 27737 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.176244 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2505) took 2448 ms. Trace:
I20260709 12:15:00.177757 27755 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.188340 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2531) took 3117 ms. Trace:
I20260709 12:15:00.213595 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:00.328334 23470 rpcz_store.cc:276] 0709 12:14:57.071023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.071439 (+   416us) service_pool.cc:224] Handling call
0709 12:14:57.071815 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:57.072186 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:57.072220 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:57.072804 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 12:14:57.166303 (+ 93499us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:14:57.166367 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:57.166429 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:58.182352 (+1015923us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:00.187996 (+2005644us) raft_consensus.cc:1685] finished
0709 12:15:00.188026 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.188275 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":221,"spinlock_wait_cycles":107304704,"thread_start_us":93163,"threads_started":1,"wal-append.queue_time_us":415187}
I20260709 12:15:00.231762 27770 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.239014 27785 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:00.246515 23321 rpcz_store.cc:276] 0709 12:14:57.425420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.425600 (+   180us) service_pool.cc:224] Handling call
0709 12:14:58.201928 (+776328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:59.942107 (+1740179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:59.942856 (+   749us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54003af550. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:14:59.943000 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784671104}
I20260709 12:15:00.248169 23316 rpcz_store.cc:276] 0709 12:14:58.635021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.931801 (+1296780us) service_pool.cc:224] Handling call
0709 12:14:59.933403 (+  1602us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:59.965196 23389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 750) took 1850 ms (client timeout 2043 ms). Trace:
I20260709 12:15:00.328218 23394 rpcz_store.cc:276] 0709 12:14:57.728264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.728465 (+   201us) service_pool.cc:224] Handling call
0709 12:14:57.728921 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.135735 (+2406814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.135766 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:00.136251 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:15:00.164299 (+ 28048us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:00.166314 (+  2015us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.173743 (+  7429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.174526 (+   783us) spinlock_profiling.cc:243] Waited 2.41 s on lock 0x7b54002af8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.174575 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:00.175921 (+  1346us) raft_consensus.cc:1685] finished
0709 12:15:00.176029 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.176176 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7703,"prepare.queue_time_us":1622,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":8624,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":8630,"spinlock_wait_cycles":5534913664,"thread_start_us":27689,"threads_started":1,"wal-append.queue_time_us":37986}
W20260709 12:15:00.331844 23389 rpcz_store.cc:269] 0709 12:14:58.114479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.963848 (+1849369us) service_pool.cc:224] Handling call
0709 12:14:59.965143 (+  1295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.248556 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.333279 27755 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:00.333717 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 85250us
I20260709 12:15:00.334210 27770 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.335011 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.335297 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.247224 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:00.335929 27625 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:00.336059 23321 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:00.336409 27625 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:00.336809 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89649.9us
I20260709 12:15:00.250092 23392 rpcz_store.cc:276] 0709 12:14:58.115353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.972726 (+1857373us) service_pool.cc:224] Handling call
0709 12:15:00.249654 (+276928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.337612 27556 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:00.249370 23320 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:15:00.250872 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2513) took 1956 ms (client timeout 1804 ms). Trace:
W20260709 12:15:00.339932 23320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2346) took 2237 ms (client timeout 2855 ms). Trace:
W20260709 12:15:00.340605 23320 rpcz_store.cc:269] 0709 12:14:58.102651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.245556 (+2142905us) service_pool.cc:224] Handling call
0709 12:15:00.339856 (+ 94300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.251803 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 885) took 1746 ms. Trace:
I20260709 12:15:00.343252 23469 rpcz_store.cc:276] 0709 12:14:58.505836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.004063 (+1498227us) service_pool.cc:224] Handling call
0709 12:15:00.004318 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.243331 (+239013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.243367 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.243413 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:00.243458 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.251646 (+  8188us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54000c8354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.251677 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.251760 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549203328}
I20260709 12:15:00.252022 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.251240 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.346108 23317 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:00.346760 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1088) took 1712 ms. Trace:
I20260709 12:15:00.245618 27811 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.249053 27814 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:00.250520 23468 rpcz_store.cc:269] 0709 12:14:58.506405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.517226 (+ 10821us) service_pool.cc:224] Handling call
0709 12:14:59.990405 (+1473179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.180857 27747 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:00.255223 23314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2340) took 2166 ms (client timeout 2882 ms). Trace:
W20260709 12:15:00.256105 23417 debug-util.cc:398] Leaking SignalData structure 0x7b080009f6a0 after lost signal to thread 23253
I20260709 12:15:00.189006 27753 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:00.353191 23314 rpcz_store.cc:269] 0709 12:14:58.088383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.246449 (+2158066us) service_pool.cc:224] Handling call
0709 12:15:00.255158 (+  8709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.254411 27774 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.355224 23313 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2343) took 2266 ms (client timeout 2165 ms). Trace:
W20260709 12:15:00.355604 23313 rpcz_store.cc:269] 0709 12:14:58.089179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.891660 (+802481us) service_pool.cc:224] Handling call
0709 12:15:00.355034 (+1463374us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.355165 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930174208}
W20260709 12:15:00.256682 23463 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 879) took 1836 ms (client timeout 1977 ms). Trace:
I20260709 12:15:00.356639 27753 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.356848 23463 rpcz_store.cc:269] 0709 12:14:58.420582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.496871 (+ 76289us) service_pool.cc:224] Handling call
0709 12:15:00.256622 (+1759751us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.257361 23390 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:00.357993 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 745) took 2245 ms. Trace:
I20260709 12:15:00.254014 27808 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.259148 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 748) took 2145 ms. Trace:
I20260709 12:15:00.358459 27722 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.358685 23388 rpcz_store.cc:276] 0709 12:14:58.114015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.257863 (+2143848us) service_pool.cc:224] Handling call
0709 12:15:00.258642 (+   779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.258816 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.258842 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.258883 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.258916 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.258969 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.259096 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.251093 27809 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:15:00.250166 27794 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:00.217320 27780 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.254990 27793 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:15:00.350104 27737 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:00.269419 23462 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:00.368558 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.026831 27818 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:00.369959 27809 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:00.370190 27818 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:00.370364 27809 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:00.371359 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:00.273883 24007 rpcz_store.cc:269] 0709 12:14:57.763560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.763717 (+   157us) service_pool.cc:224] Handling call
0709 12:15:00.043346 (+2279629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.277064 27805 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.279433 24016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1022) took 2145 ms (client timeout 1945 ms). Trace:
W20260709 12:15:00.279582 24012 rpcz_store.cc:269] 0709 12:14:58.270395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.052249 (+1781854us) service_pool.cc:224] Handling call
0709 12:15:00.053273 (+  1024us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.283280 24014 rpcz_store.cc:269] 0709 12:14:58.135275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.071882 (+1936607us) service_pool.cc:224] Handling call
0709 12:15:00.073233 (+  1351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.280745 27321 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.283916 23467 rpcz_store.cc:276] 0709 12:14:58.505296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.999914 (+1494618us) service_pool.cc:224] Handling call
0709 12:15:00.000188 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.073972 (+ 73784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.074026 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.074078 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:00.074116 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.075934 (+  1818us) spinlock_profiling.cc:243] Waited 73.6 ms on lock 0x7b54002a03d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.075994 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.076171 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169170176}
I20260709 12:15:00.374356 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.289680 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:00.289991 27804 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.290390 27666 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.291220 27720 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.383988 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.384063 27720 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.385586 27321 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.293085 27647 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.293630 27580 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.386339 27720 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.292537 27784 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.285585 27813 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.386495 27580 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.387081 27784 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.298525 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:00.294314 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2222) took 1502 ms. Trace:
I20260709 12:15:00.298790 27782 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.305804 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.306416 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:00.310851 27810 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.390348 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1024) took 2253 ms (client timeout 2855 ms). Trace:
W20260709 12:15:00.390762 24010 rpcz_store.cc:269] 0709 12:14:58.136499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.288032 (+2151533us) service_pool.cc:224] Handling call
0709 12:15:00.390290 (+102258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.367770 27737 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.390269 23393 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:00.391661 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.391994 23393 raft_consensus.cc:614] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:00.392678 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.392872 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.393508 24010 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:00.393824 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.394328 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.142925 27685 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.312114 27817 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:15:00.396616 27804 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:00.399259 27804 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.313827 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.314803 24008 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:00.316708 27637 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:00.281371 23217 rpcz_store.cc:269] 0709 12:14:41.364668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:41.365049 (+   381us) service_pool.cc:224] Handling call
0709 12:14:41.671557 (+306508us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:14:41.720290 (+ 48733us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a6550640 00007ff0af427846 00007ff0af53190e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.063391 (+18343101us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:57.527696 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:57.684807 (+157111us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:57.712028 (+ 27221us) write_op.cc:435] Acquired schema lock
0709 12:14:57.712065 (+    37us) tablet.cc:662] Decoding operations
0709 12:14:57.774720 (+ 62655us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:57.782204 (+  7484us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:57.782259 (+    55us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:14:57.794833 (+ 12574us) tablet.cc:701] Row locks acquired
0709 12:14:57.794887 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:14:57.795122 (+   235us) write_op.cc:270] Start()
0709 12:14:57.795261 (+   139us) write_op.cc:276] Timestamp: P: 1783599297795032 usec, L: 0
0709 12:14:57.795287 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:14:58.009371 (+214084us) log.cc:844] Serialized 6766 byte log entry
0709 12:14:58.082416 (+ 73045us) op_driver.cc:464] REPLICATION: finished
0709 12:14:58.091786 (+  9370us) write_op.cc:301] APPLY: starting
0709 12:14:58.091890 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:58.144817 (+ 52927us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:58.144951 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:58.176992 (+ 32041us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:58.177296 (+   304us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 12:14:58.177342 (+    46us) write_op.cc:312] APPLY: finished
0709 12:14:58.177767 (+   425us) log.cc:844] Serialized 156 byte log entry
0709 12:14:58.177957 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:58.178129 (+   172us) write_op.cc:454] Released schema lock
0709 12:14:58.178179 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:58.361923 (+183744us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a966fe28 00007ff0ad0aa581 00007ff0ad0a9d58 00007ff0ad0ae9f1 00007ff0ad0ae9a9 00007ff0ad0ae939 00007ff0ad0ae901 00007ff0ad0adb2d 0000000000388634 0000000000388479
0709 12:14:58.362170 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:58.362203 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3932800,"reported_tablets":47,"spinlock_wait_cycles":665192064,"child_traces":[["op",{"apply.queue_time_us":9118,"apply.run_cpu_time_us":25543,"apply.run_wall_time_us":86815,"cfile_cache_hit":66,"cfile_cache_hit_bytes":69036,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":2,"lbm_read_time_us":23840,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":17,"prepare.queue_time_us":619281,"prepare.run_cpu_time_us":14946,"prepare.run_wall_time_us":849426,"raft.queue_time_us":51009,"raft.run_cpu_time_us":2459,"raft.run_wall_time_us":11495,"replication_time_us":286966,"spinlock_wait_cycles":490590464,"thread_start_us":373525,"threads_started":4,"wal-append.queue_time_us":10291,"wal-append.run_cpu_time_us":18810,"wal-append.run_wall_time_us":1158859}]]}
I20260709 12:15:00.316630 27823 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.401170 23319 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:15:00.324506 23391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2507) took 2594 ms (client timeout 1640 ms). Trace:
W20260709 12:15:00.401711 23391 rpcz_store.cc:269] 0709 12:14:57.730185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.730303 (+   118us) service_pool.cc:224] Handling call
0709 12:15:00.324329 (+2594026us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b5400190ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.324429 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4689533312}
I20260709 12:15:00.402786 27811 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.402844 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:00.403597 23391 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.326346 24013 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:00.405332 27637 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.406121 23314 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:15:00.406522 24013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 983) took 2606 ms (client timeout 2995 ms). Trace:
I20260709 12:15:00.406484 27811 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:00.407176 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2337) took 2319 ms. Trace:
W20260709 12:15:00.407074 24013 rpcz_store.cc:269] 0709 12:14:57.799795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.802008 (+  2213us) service_pool.cc:224] Handling call
0709 12:15:00.406348 (+2604340us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54006bdad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.406471 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515864448}
I20260709 12:15:00.407583 27817 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:00.391547 27737 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:00.333323 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 749) took 2219 ms. Trace:
I20260709 12:15:00.336939 27625 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:00.005460 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 882) took 1500 ms. Trace:
I20260709 12:15:00.250525 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:00.413028 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.412580 27580 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:00.413622 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 163156us
I20260709 12:15:00.414319 27823 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.384138 27666 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.415019 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.416718 24010 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:00.416999 27666 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.338157 23392 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:15:00.417645 27782 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.417836 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2510) took 2147 ms. Trace:
I20260709 12:15:00.418071 27782 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.418453 27782 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.418226 23392 rpcz_store.cc:276] 0709 12:14:58.270652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.337535 (+2066883us) service_pool.cc:224] Handling call
0709 12:15:00.337854 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.417583 (+ 79729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.417780 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.419441 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.420598 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.421106 23392 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:00.331291 27785 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:00.277179 27806 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.337292 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:00.467856 23321 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:00.339882 23387 rpcz_store.cc:269] 0709 12:14:58.294503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.960891 (+1666388us) service_pool.cc:224] Handling call
0709 12:15:00.250798 (+289907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.251449 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 746) took 2138 ms. Trace:
I20260709 12:15:00.342882 23394 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:00.468520 23396 rpcz_store.cc:276] 0709 12:14:58.113069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.988387 (+1875318us) service_pool.cc:224] Handling call
0709 12:15:00.251389 (+263002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.469151 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2509) took 2205 ms. Trace:
I20260709 12:15:00.469575 23394 rpcz_store.cc:276] 0709 12:14:58.263574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.342301 (+2078727us) service_pool.cc:224] Handling call
0709 12:15:00.342581 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.468858 (+126277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.469090 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.457854 27549 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.470782 23315 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:00.342458 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2336) took 2255 ms. Trace:
I20260709 12:15:00.347170 23317 rpcz_store.cc:276] 0709 12:14:58.634779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.250998 (+1616219us) service_pool.cc:224] Handling call
0709 12:15:00.346689 (+ 95691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.469662 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.345031 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.471729 23315 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2335) took 2384 ms (client timeout 2774 ms). Trace:
W20260709 12:15:00.354318 23417 debug-util.cc:398] Leaking SignalData structure 0x7b080009fc60 after lost signal to thread 23345
I20260709 12:15:00.354125 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.245781 27812 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.359773 23461 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:00.472128 23315 rpcz_store.cc:269] 0709 12:14:58.087034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.244074 (+2157040us) service_pool.cc:224] Handling call
0709 12:15:00.471559 (+227485us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400296154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.471678 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518968064}
I20260709 12:15:00.472060 23469 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:00.473042 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.473373 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.473973 23315 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:15:00.473989 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.473143 23417 debug-util.cc:398] Leaking SignalData structure 0x7b080018aec0 after lost signal to thread 23420
I20260709 12:15:00.474673 23469 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:00.475061 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:00.475474 23417 debug-util.cc:398] Leaking SignalData structure 0x7b08003e2600 after lost signal to thread 23494
I20260709 12:15:00.475458 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.475803 23315 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:00.360858 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 746) took 2588 ms (client timeout 1766 ms). Trace:
I20260709 12:15:00.358307 23390 rpcz_store.cc:276] 0709 12:14:58.112865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.256894 (+2144029us) service_pool.cc:224] Handling call
0709 12:15:00.257201 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.357740 (+100539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.357930 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.359757 23312 raft_consensus.cc:2376] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15. Re-sending same reply.
I20260709 12:15:00.363186 23316 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:15:00.477118 24011 rpcz_store.cc:269] 0709 12:14:57.772755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.772951 (+   196us) service_pool.cc:224] Handling call
0709 12:15:00.360692 (+2587741us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540011d5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.360790 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378287488}
I20260709 12:15:00.477764 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2339) took 2389 ms. Trace:
I20260709 12:15:00.471122 23320 rpcz_store.cc:276] 0709 12:14:58.087334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.341456 (+2254122us) service_pool.cc:224] Handling call
0709 12:15:00.341818 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.342034 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.342072 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.342134 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:00.342179 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.342255 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.342392 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.478142 23316 rpcz_store.cc:276] 0709 12:14:58.088106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.361650 (+2273544us) service_pool.cc:224] Handling call
0709 12:15:00.362873 (+  1223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.477510 (+114637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.477709 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.478772 23396 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:00.266675 27744 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.366335 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:00.368487 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.270706 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002a2380 after lost signal to thread 23420
I20260709 12:15:00.362233 27794 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:00.271862 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 245141us
I20260709 12:15:00.351864 27814 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.375203 24015 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:00.493243 23312 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2347) took 1857 ms (client timeout 1703 ms). Trace:
W20260709 12:15:00.372282 24016 rpcz_store.cc:269] 0709 12:14:58.134312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.134457 (+   145us) service_pool.cc:224] Handling call
0709 12:15:00.279357 (+2144900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.359122 27722 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.381896 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.500090 23468 raft_consensus.cc:2410] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:00.382088 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.359057 27747 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.371104 27818 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.385489 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.388257 23462 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:00.350459 27729 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:15:00.388854 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:00.388130 23464 rpcz_store.cc:276] 0709 12:14:58.792179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.995269 (+1203090us) service_pool.cc:224] Handling call
0709 12:15:00.294151 (+298882us) spinlock_profiling.cc:243] Waited 77.9 ms on lock 0x7b5400225bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.294258 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179113344}
W20260709 12:15:00.389695 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 91264.3us
I20260709 12:15:00.510501 23467 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.131873 27659 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.392886 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.514977 23393 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:15:00.394419 24012 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:00.399600 27804 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:00.400630 24008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 749) took 2159 ms (client timeout 1804 ms). Trace:
I20260709 12:15:00.323592 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2504) took 2762 ms. Trace:
I20260709 12:15:00.515935 23395 rpcz_store.cc:276] 0709 12:14:57.560941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:57.561065 (+   124us) service_pool.cc:224] Handling call
0709 12:14:57.561375 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.312683 (+2751308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.323314 (+ 10631us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x7b5400058554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.323534 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5887679232}
I20260709 12:15:00.517674 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:00.518301 23395 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.511389 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.519019 23316 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:00.408272 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.519887 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.520866 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.521411 23316 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:15:00.521248 27625 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.330260 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.391474 27784 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.525702 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:00.531611 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.410537 23389 rpcz_store.cc:276] 0709 12:14:58.114269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.332522 (+2218253us) service_pool.cc:224] Handling call
0709 12:15:00.332830 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.333026 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.333052 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.333092 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.333123 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.333177 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.333271 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.533110 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.533280 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.371183 27809 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.337980 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.421944 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2341) took 2333 ms. Trace:
I20260709 12:15:00.304024 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:00.580397 27752 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:00.588641 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 284679us
I20260709 12:15:00.590063 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:00.400849 27813 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.422580 27436 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.457876 27782 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.591075 27436 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.592444 27782 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.421969 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 751) took 2180 ms. Trace:
I20260709 12:15:00.593042 24010 rpcz_store.cc:276] 0709 12:14:58.241309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.420651 (+2179342us) service_pool.cc:224] Handling call
0709 12:15:00.420968 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.421641 (+   673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.421673 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.421715 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:00.421749 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.421804 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.421918 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101}
I20260709 12:15:00.593797 27436 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.462966 27808 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:00.443740 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.408008 27817 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:00.455713 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.412868 23465 rpcz_store.cc:276] 0709 12:14:58.505035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.524318 (+ 19283us) service_pool.cc:224] Handling call
0709 12:15:00.005291 (+1480973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.597414 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.598002 23465 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:00.598769 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.599269 23465 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:00.466502 23391 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:00.338011 27556 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.349046 23218 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:00.560081 27818 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.471994 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.602733 23317 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
W20260709 12:15:00.360338 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5989981529999113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 12:15:00.603592 23287 connection.cc:256] RPC call timeout handler was delayed by -1.2050309179999203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 12:15:00.603974 23287 connection.cc:256] RPC call timeout handler was delayed by -1.2050309179999203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 12:15:00.604363 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1348309179998068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 12:15:00.604628 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0214309179998509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 12:15:00.604898 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0214309179998509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
I20260709 12:15:00.339196 27586 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:00.476660 23417 debug-util.cc:398] Leaking SignalData structure 0x7b08002a4160 after lost signal to thread 24040
W20260709 12:15:00.611689 23417 debug-util.cc:398] Leaking SignalData structure 0x7b0800323d40 after lost signal to thread 24041
I20260709 12:15:00.476649 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.477638 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.617942 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1085) took 2315 ms. Trace:
W20260709 12:15:00.494748 23312 rpcz_store.cc:269] 0709 12:14:58.635748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:59.929364 (+1293616us) service_pool.cc:224] Handling call
0709 12:15:00.481316 (+551952us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:00.493149 (+ 11833us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257458048}
W20260709 12:15:00.482952 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae080 after lost signal to thread 23494
I20260709 12:15:00.618294 23315 rpcz_store.cc:276] 0709 12:14:58.302631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.617180 (+2314549us) service_pool.cc:224] Handling call
0709 12:15:00.617500 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.617670 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.617694 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.617729 (+    35us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:00.617757 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.617806 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.617893 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.494102 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.496377 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.481745 23388 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.480690 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.504381 23321 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:00.508625 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.621131 23317 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:15:00.621376 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.511804 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:00.621999 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2344) took 2532 ms. Trace:
I20260709 12:15:00.522481 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.522704 23470 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:00.622498 23317 rpcz_store.cc:276] 0709 12:14:58.101969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.620507 (+2518538us) service_pool.cc:224] Handling call
0709 12:15:00.620823 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.621680 (+   857us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.621932 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.623736 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.624539 23470 raft_consensus.cc:2410] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:15:00.515766 24008 rpcz_store.cc:269] 0709 12:14:58.240823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:58.751699 (+510876us) service_pool.cc:224] Handling call
0709 12:15:00.400559 (+1648860us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.407682 23314 rpcz_store.cc:276] 0709 12:14:58.087618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.353963 (+2266345us) service_pool.cc:224] Handling call
0709 12:15:00.354256 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.406461 (+ 52205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.406968 (+   507us) spinlock_profiling.cc:243] Waited 51.5 ms on lock 0x7b54002422d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:00.407123 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118559616}
I20260709 12:15:00.536203 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.536831 24007 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:15:00.625876 23396 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:00.543800 23461 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:00.545948 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:00.515403 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:00.527848 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:00.548416 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
W20260709 12:15:00.550136 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 212222us
I20260709 12:15:00.627362 23389 raft_consensus.cc:606] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.627729 23389 raft_consensus.cc:614] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:00.629038 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1087) took 1994 ms. Trace:
I20260709 12:15:00.629243 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.549965 23319 rpcz_store.cc:276] 0709 12:14:58.088637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.410144 (+2321507us) service_pool.cc:224] Handling call
0709 12:15:00.421404 (+ 11260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.421594 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.421622 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.421678 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:00.421724 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.421785 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.421891 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.630899 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1090) took 1995 ms. Trace:
I20260709 12:15:00.629462 23312 rpcz_store.cc:276] 0709 12:14:58.634522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.628319 (+1993797us) service_pool.cc:224] Handling call
0709 12:15:00.628591 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.628765 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.628789 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.628827 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.628855 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.628904 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.628989 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.606695 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:00.631286 23315 rpcz_store.cc:276] 0709 12:14:58.635262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.630174 (+1994912us) service_pool.cc:224] Handling call
0709 12:15:00.630443 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.630620 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.630648 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.630686 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.630717 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.630766 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.630852 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.552155 24011 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:00.577399 23393 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:00.633311 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2514) took 2337 ms. Trace:
I20260709 12:15:00.633679 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2348) took 1997 ms. Trace:
I20260709 12:15:00.579090 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.532369 27744 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:00.532276 23468 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:00.581631 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.638079 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:00.586375 23467 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:00.588542 27752 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:00.470597 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.599807 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.644304 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:00.609666 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2223) took 1817 ms. Trace:
I20260709 12:15:00.644799 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1094) took 1895 ms. Trace:
I20260709 12:15:00.620143 23320 raft_consensus.cc:2410] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:15:00.633654 23393 rpcz_store.cc:276] 0709 12:14:58.295600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.568786 (+2273186us) service_pool.cc:224] Handling call
0709 12:15:00.576957 (+  8171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.633096 (+ 56139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.633260 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.634238 23317 rpcz_store.cc:276] 0709 12:14:58.636010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.632824 (+1996814us) service_pool.cc:224] Handling call
0709 12:15:00.633085 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.633237 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.633263 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.633302 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:00.633334 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.633387 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.633627 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.634202 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1091) took 1998 ms. Trace:
I20260709 12:15:00.631439 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2349) took 1995 ms. Trace:
I20260709 12:15:00.643028 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1093) took 1929 ms. Trace:
I20260709 12:15:00.645301 23315 rpcz_store.cc:276] 0709 12:14:58.749745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.643980 (+1894235us) service_pool.cc:224] Handling call
0709 12:15:00.644247 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.644411 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.644454 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.644493 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.644524 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.644573 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.644661 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.646059 23312 rpcz_store.cc:276] 0709 12:14:58.713962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.642365 (+1928403us) service_pool.cc:224] Handling call
0709 12:15:00.642602 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.642765 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.642787 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.642820 (+    33us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:00.642853 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.642894 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.642983 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.648699 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 133541us
W20260709 12:15:00.619482 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080034ce60 after lost signal to thread 24007
W20260709 12:15:00.670575 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080036e2e0 after lost signal to thread 24009
W20260709 12:15:00.670854 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080032f7c0 after lost signal to thread 24010
I20260709 12:15:00.642973 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1092) took 1929 ms. Trace:
I20260709 12:15:00.649991 23467 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:00.645849 23314 rpcz_store.cc:276] 0709 12:14:58.636242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.630721 (+1994479us) service_pool.cc:224] Handling call
0709 12:15:00.631009 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.631158 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.631182 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.631219 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:00.631252 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.631300 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.631388 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.665970 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 889) took 1804 ms. Trace:
I20260709 12:15:00.665970 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 888) took 1804 ms. Trace:
I20260709 12:15:00.646483 23321 rpcz_store.cc:276] 0709 12:14:58.635494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.633504 (+1998010us) service_pool.cc:224] Handling call
0709 12:15:00.633757 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.633908 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.633943 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.633981 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.634012 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.634062 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.634150 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.632537 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:00.667865 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 140087us
I20260709 12:15:00.668057 27465 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:00.607342 27666 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:00.638521 23218 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 8 to 10, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:00.601022 27809 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:15:00.706372 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73946.7us
I20260709 12:15:00.710876 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 890) took 1848 ms. Trace:
I20260709 12:15:00.712527 23461 rpcz_store.cc:276] 0709 12:14:58.862737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.710134 (+1847397us) service_pool.cc:224] Handling call
0709 12:15:00.710424 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.710585 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.710610 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.710651 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.710685 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.710734 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.710824 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.676302 23319 rpcz_store.cc:276] 0709 12:14:58.713586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.642202 (+1928616us) service_pool.cc:224] Handling call
0709 12:15:00.642514 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.642691 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.642716 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.642754 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:00.642784 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.642833 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.642923 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.595238 27817 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.718348 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 891) took 1854 ms. Trace:
I20260709 12:15:00.586997 27794 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.718824 23461 rpcz_store.cc:276] 0709 12:14:58.863706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.717207 (+1853501us) service_pool.cc:224] Handling call
0709 12:15:00.717874 (+   667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.718057 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.718081 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.718121 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.718157 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.718206 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.718299 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.604123 27556 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:00.692376 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2229) took 1898 ms. Trace:
I20260709 12:15:00.676417 23465 rpcz_store.cc:276] 0709 12:14:58.861369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.665115 (+1803746us) service_pool.cc:224] Handling call
0709 12:15:00.665506 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.665673 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.665699 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.665739 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.665770 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.665818 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.665911 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39808}
I20260709 12:15:00.692541 23468 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:15:00.720467 23467 rpcz_store.cc:276] 0709 12:14:58.794039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.649315 (+1855276us) service_pool.cc:224] Handling call
0709 12:15:00.649699 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.676338 (+ 26639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.692312 (+ 15974us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41600}
I20260709 12:15:00.721233 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2226) took 1928 ms. Trace:
I20260709 12:15:00.721678 23468 rpcz_store.cc:276] 0709 12:14:58.793200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.648628 (+1855428us) service_pool.cc:224] Handling call
0709 12:15:00.692199 (+ 43571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.720975 (+ 28776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.721169 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.722201 23467 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:00.723068 23468 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:00.723726 23467 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:15:00.693512 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2350) took 1905 ms. Trace:
I20260709 12:15:00.629916 27436 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.725095 23467 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:15:00.642035 27681 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:00.693111 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1096) took 1934 ms. Trace:
I20260709 12:15:00.718144 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:00.696319 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1095) took 1945 ms. Trace:
I20260709 12:15:00.732995 23316 rpcz_store.cc:276] 0709 12:14:58.750487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.670564 (+1920077us) service_pool.cc:224] Handling call
0709 12:15:00.695814 (+ 25250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.695972 (+   158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.696020 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.696061 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:00.696091 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.696147 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.696253 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:00.734678 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.695948 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.677858 23470 rpcz_store.cc:276] 0709 12:14:58.862018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.665123 (+1803105us) service_pool.cc:224] Handling call
0709 12:15:00.665506 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.665673 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.665699 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.665739 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:00.665768 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.665818 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.665911 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46208}
I20260709 12:15:00.671687 27816 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:00.745312 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:00.745837 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.619647 27813 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:00.655470 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:00.750242 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94845.4us
I20260709 12:15:00.648859 27752 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:00.704324 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:00.713256 27737 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.580782 27785 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:00.676676 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003848a0 after lost signal to thread 24011
I20260709 12:15:00.725620 27436 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.671739 27661 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:00.726262 27625 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:00.727339 27722 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.727568 23468 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:15:00.724422 23314 rpcz_store.cc:276] 0709 12:14:58.787908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.679180 (+1891272us) service_pool.cc:224] Handling call
0709 12:15:00.692986 (+ 13806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.693162 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.693184 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.693222 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:00.693257 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.693312 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.693414 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.706351 27465 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.789808 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.790580 23314 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:00.732286 23321 rpcz_store.cc:276] 0709 12:14:58.758827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.676611 (+1917784us) service_pool.cc:224] Handling call
0709 12:15:00.692640 (+ 16029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.692812 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.692836 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.692877 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.692906 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.692967 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.693057 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:00.791894 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.791925 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.732811 23319 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.637495 27744 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:00.809787 27785 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:00.810081 27744 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.811367 27744 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:00.812474 27722 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:00.713018 27549 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:00.732132 27681 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.739426 23388 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:15:00.739815 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44036.7us
I20260709 12:15:00.732729 23218 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:00.733832 27794 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:00.766996 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62752.2us
I20260709 12:15:00.824536 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:00.711021 27647 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:00.740025 27817 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.768419 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:00.834865 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66728.6us
I20260709 12:15:00.779453 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:00.780220 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080035fb80 after lost signal to thread 24012
W20260709 12:15:00.836045 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0160 after lost signal to thread 24013
W20260709 12:15:00.836339 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800395ec0 after lost signal to thread 24016
I20260709 12:15:00.793831 27465 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.787721 23463 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:00.731348 23464 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:00.794382 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:00.795572 27829 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:15:00.797207 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.769006 27816 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:00.804436 23313 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:00.805532 27737 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:00.848489 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 51371.8us
I20260709 12:15:00.848678 27737 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:00.849160 27737 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.793962 27661 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:00.849915 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.850091 27661 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:00.851236 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:00.851280 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.852658 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.814823 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.853699 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.853994 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.854432 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.855046 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.855651 23321 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:00.856690 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.855670 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:00.819552 23391 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:00.778262 27770 raft_consensus.cc:2764] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:00.864840 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:00.829025 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:00.829756 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.831095 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8ea0 after lost signal to thread 22110
I20260709 12:15:00.866065 23470 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:15:00.866681 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6b80 after lost signal to thread 23253
W20260709 12:15:00.867208 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08001a8f40 after lost signal to thread 23287
I20260709 12:15:00.867237 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.867938 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800347f60 after lost signal to thread 23315
W20260709 12:15:00.868214 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003d5fe0 after lost signal to thread 23316
W20260709 12:15:00.868412 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800348720 after lost signal to thread 23317
W20260709 12:15:00.868636 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800388c60 after lost signal to thread 23318
W20260709 12:15:00.868963 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6ac0 after lost signal to thread 23321
I20260709 12:15:00.826632 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:00.836863 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800277980 after lost signal to thread 24040
W20260709 12:15:00.870275 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08001d2c40 after lost signal to thread 24041
W20260709 12:15:00.870661 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080031eb60 after lost signal to thread 27200
W20260709 12:15:00.870934 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9320 after lost signal to thread 27321
W20260709 12:15:00.871217 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800390000 after lost signal to thread 27401
W20260709 12:15:00.871487 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003bdc40 after lost signal to thread 27434
W20260709 12:15:00.871769 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080034b820 after lost signal to thread 27465
W20260709 12:15:00.872053 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002bc220 after lost signal to thread 27480
W20260709 12:15:00.872344 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080027b5e0 after lost signal to thread 27532
W20260709 12:15:00.872628 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002935a0 after lost signal to thread 27549
W20260709 12:15:00.872917 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003e23a0 after lost signal to thread 27554
W20260709 12:15:00.873189 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08001be160 after lost signal to thread 27556
I20260709 12:15:00.745988 27659 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:15:00.876214 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800265640 after lost signal to thread 23345
W20260709 12:15:00.876766 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080026a940 after lost signal to thread 23362
I20260709 12:15:00.876847 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:00.877651 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003501e0 after lost signal to thread 23387
W20260709 12:15:00.877920 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080032bde0 after lost signal to thread 23388
W20260709 12:15:00.837343 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:00.878197 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800247880 after lost signal to thread 23389
W20260709 12:15:00.878476 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800396c00 after lost signal to thread 23390
W20260709 12:15:00.878700 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003915a0 after lost signal to thread 23392
I20260709 12:15:00.842777 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:00.878959 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800267f00 after lost signal to thread 23396
I20260709 12:15:00.844476 24008 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:00.827994 27681 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.859090 23321 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:00.861683 23316 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:00.862099 27737 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.846889 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.882730 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.832473 27647 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:00.645118 23466 rpcz_store.cc:276] 0709 12:14:58.792449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.268920 (+1476471us) service_pool.cc:224] Handling call
0709 12:15:00.269189 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.269384 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:00.367174 (+ 97790us) log.cc:844] Serialized 14 byte log entry
0709 12:15:00.609284 (+242110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.609338 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:00.609387 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.609481 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:00.609604 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7251,"prepare.queue_time_us":87811,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":17244,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12001,"thread_start_us":329207,"threads_started":2}
W20260709 12:15:00.883513 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6a40 after lost signal to thread 27558
W20260709 12:15:00.957890 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080031a340 after lost signal to thread 27559
I20260709 12:15:00.887555 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.958424 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003d6ee0 after lost signal to thread 27568
W20260709 12:15:00.958812 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800148900 after lost signal to thread 27579
W20260709 12:15:00.959118 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003c5a40 after lost signal to thread 27580
W20260709 12:15:00.879513 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002d8fe0 after lost signal to thread 23420
I20260709 12:15:00.901607 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:00.959888 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800344e20 after lost signal to thread 23434
I20260709 12:15:00.916396 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:00.960763 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.958222 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.926746 23218 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 5 to 8, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:00.959417 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003336e0 after lost signal to thread 27586
I20260709 12:15:00.879211 24011 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
W20260709 12:15:00.962386 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003e2420 after lost signal to thread 27625
W20260709 12:15:00.962679 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08001493e0 after lost signal to thread 27650
I20260709 12:15:00.962772 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.915113 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.962932 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003fe420 after lost signal to thread 27652
W20260709 12:15:00.960350 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080034ffa0 after lost signal to thread 23436
I20260709 12:15:00.839515 27793 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:00.895565 24014 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:00.963783 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080029f9c0 after lost signal to thread 27654
I20260709 12:15:00.883888 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:00.868909 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.964573 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800198ba0 after lost signal to thread 23462
I20260709 12:15:00.855182 27816 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:00.856113 27777 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:15:00.976662 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6740 after lost signal to thread 27656
I20260709 12:15:00.973996 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:00.977300 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:00.977603 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.979589 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:00.978258 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800369380 after lost signal to thread 23463
W20260709 12:15:00.980729 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6720 after lost signal to thread 23464
W20260709 12:15:00.982618 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800396c60 after lost signal to thread 23465
W20260709 12:15:00.982967 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800152b80 after lost signal to thread 23466
W20260709 12:15:00.993788 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003fec00 after lost signal to thread 27659
W20260709 12:15:00.994282 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8880 after lost signal to thread 27661
I20260709 12:15:00.980579 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.985719 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:00.994691 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003cede0 after lost signal to thread 27662
I20260709 12:15:00.995427 24011 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:00.875077 27549 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:00.984889 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:00.997421 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.863392 27722 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:00.995162 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:00.928465 27747 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:15:00.988911 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800333080 after lost signal to thread 23467
I20260709 12:15:00.999574 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:01.000152 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080012bb20 after lost signal to thread 23468
I20260709 12:15:00.885624 27685 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:00.981402 27816 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:00.928138 27770 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:00.965005 27647 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.843374 27579 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.138s
W20260709 12:15:00.995638 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800202420 after lost signal to thread 27666
W20260709 12:15:01.038012 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800387640 after lost signal to thread 27667
I20260709 12:15:00.999078 27810 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:00.884773 27757 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 12:15:01.003829 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.003218 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:01.001132 27436 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:00.974939 27666 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.009860 27752 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:01.000466 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9aa0 after lost signal to thread 23469
W20260709 12:15:01.049453 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002082e0 after lost signal to thread 23494
W20260709 12:15:01.050407 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4080 after lost signal to thread 24007
W20260709 12:15:01.050766 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003a0300 after lost signal to thread 24009
W20260709 12:15:01.051031 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4340 after lost signal to thread 24010
W20260709 12:15:01.051273 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800207c20 after lost signal to thread 24011
W20260709 12:15:01.051513 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080024b9a0 after lost signal to thread 24012
W20260709 12:15:01.051759 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080017f400 after lost signal to thread 24013
W20260709 12:15:01.052019 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080022bd40 after lost signal to thread 24014
W20260709 12:15:01.052270 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9be0 after lost signal to thread 24015
W20260709 12:15:01.052523 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080026d380 after lost signal to thread 24016
W20260709 12:15:01.053181 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003d8120 after lost signal to thread 24040
W20260709 12:15:01.038370 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003072a0 after lost signal to thread 27670
W20260709 12:15:01.054312 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080026a2a0 after lost signal to thread 27681
W20260709 12:15:01.054630 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080032f880 after lost signal to thread 27686
W20260709 12:15:01.054893 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08002df840 after lost signal to thread 27691
W20260709 12:15:01.055193 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003c9960 after lost signal to thread 27694
W20260709 12:15:01.055472 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003aa280 after lost signal to thread 27700
W20260709 12:15:01.055809 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080031d100 after lost signal to thread 27708
W20260709 12:15:01.056114 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800313500 after lost signal to thread 27710
W20260709 12:15:01.056376 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003881e0 after lost signal to thread 27715
I20260709 12:15:01.038443 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:01.029156 27814 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.521s	user 0.004s	sys 0.004s
I20260709 12:15:01.040596 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.044067 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2338) took 2956 ms. Trace:
I20260709 12:15:01.058493 23313 rpcz_store.cc:276] 0709 12:14:58.087866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.356336 (+2268470us) service_pool.cc:224] Handling call
0709 12:15:00.356612 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.853493 (+496881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.043715 (+190222us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002756d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:01.043983 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1029467008}
I20260709 12:15:01.046432 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:00.996732 27777 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:01.002489 27747 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:01.059813 24010 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:00.904495 27804 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:00.976383 27793 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:01.173889 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:01.174790 23467 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:01.182662 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:01.045226 27685 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:01.061017 27813 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:01.062062 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003eafe0 after lost signal to thread 27716
W20260709 12:15:01.189294 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080037cfe0 after lost signal to thread 27718
W20260709 12:15:01.189715 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08003bf400 after lost signal to thread 27721
W20260709 12:15:01.189996 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080021f520 after lost signal to thread 27722
W20260709 12:15:01.190291 23342 debug-util.cc:398] Leaking SignalData structure 0x7b080035e4a0 after lost signal to thread 27727
W20260709 12:15:01.190572 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800374880 after lost signal to thread 27729
W20260709 12:15:01.190915 23342 debug-util.cc:398] Leaking SignalData structure 0x7b08001076c0 after lost signal to thread 27744
W20260709 12:15:01.191339 23342 debug-util.cc:398] Leaking SignalData structure 0x7b0800345660 after lost signal to thread 27764
W20260709 12:15:01.062636 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002caa00 after lost signal to thread 24041
W20260709 12:15:01.193691 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080037a4c0 after lost signal to thread 27200
I20260709 12:15:00.983822 27401 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:01.063236 23321 raft_consensus.cc:2410] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:15:01.000727 27549 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:01.146776 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.173161 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.185731 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:01.186949 23319 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:15:01.201920 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003e7860 after lost signal to thread 27321
W20260709 12:15:01.207219 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:01.211234 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9240 after lost signal to thread 27348
W20260709 12:15:01.211565 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08001b5520 after lost signal to thread 27401
W20260709 12:15:01.211845 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800339620 after lost signal to thread 27434
W20260709 12:15:01.212142 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080034dd00 after lost signal to thread 27465
W20260709 12:15:01.212420 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800336140 after lost signal to thread 27480
W20260709 12:15:01.212687 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800350140 after lost signal to thread 27532
W20260709 12:15:01.212940 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c5360 after lost signal to thread 27549
W20260709 12:15:01.213199 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800340200 after lost signal to thread 27552
I20260709 12:15:01.107168 27747 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:01.213865 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800191d40 after lost signal to thread 27554
W20260709 12:15:01.218279 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002d3200 after lost signal to thread 27556
W20260709 12:15:01.218690 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003cf580 after lost signal to thread 27558
W20260709 12:15:01.219033 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c7f00 after lost signal to thread 27559
W20260709 12:15:01.219367 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003d7220 after lost signal to thread 27568
W20260709 12:15:01.219722 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003eeb20 after lost signal to thread 27579
W20260709 12:15:01.220052 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800288f80 after lost signal to thread 27580
I20260709 12:15:01.222631 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:01.182047 27793 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:01.225189 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.208094 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.185151 27659 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:01.243029 23982 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.010s	sys 0.030s
W20260709 12:15:01.243598 23982 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.010s	sys 0.030s
I20260709 12:15:01.238490 24009 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:01.243961 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 706497us
I20260709 12:15:01.245407 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1098) took 1412 ms. Trace:
I20260709 12:15:01.245929 23319 rpcz_store.cc:276] 0709 12:14:59.832667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.244468 (+1411801us) service_pool.cc:224] Handling call
0709 12:15:01.244861 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.245069 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.245101 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.245142 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:01.245170 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.245218 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.245327 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:01.055320 27666 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:01.189015 27813 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:01.251936 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0f80 after lost signal to thread 27586
W20260709 12:15:01.266676 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080036eaa0 after lost signal to thread 27625
W20260709 12:15:01.266995 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3de0 after lost signal to thread 27647
W20260709 12:15:01.267381 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800149ee0 after lost signal to thread 27650
W20260709 12:15:01.267612 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000d1ee0 after lost signal to thread 27652
W20260709 12:15:01.268352 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4c60 after lost signal to thread 27654
I20260709 12:15:01.064329 27814 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.257701 24010 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:01.215766 27549 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:01.278756 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:01.279618 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:01.280691 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:01.281399 23466 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:01.282572 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:01.269140 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0020 after lost signal to thread 27656
W20260709 12:15:01.283741 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3d80 after lost signal to thread 27659
I20260709 12:15:01.237695 27808 raft_consensus.cc:2764] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:15:01.284418 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080039fd60 after lost signal to thread 27661
W20260709 12:15:01.303892 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003aa720 after lost signal to thread 27662
W20260709 12:15:01.304315 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002e62e0 after lost signal to thread 27666
W20260709 12:15:01.304630 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800405140 after lost signal to thread 27667
W20260709 12:15:01.266937 27818 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.000s
I20260709 12:15:01.305065 23468 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.048394 27770 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:01.306933 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800398640 after lost signal to thread 27668
I20260709 12:15:01.269691 27813 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:01.313124 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800364220 after lost signal to thread 27670
W20260709 12:15:01.318840 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003419a0 after lost signal to thread 27681
W20260709 12:15:01.319144 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800387ac0 after lost signal to thread 27685
W20260709 12:15:01.319371 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800346940 after lost signal to thread 27686
W20260709 12:15:01.319576 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800375900 after lost signal to thread 27691
W20260709 12:15:01.319789 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080034d640 after lost signal to thread 27694
W20260709 12:15:01.320004 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003a0120 after lost signal to thread 27700
W20260709 12:15:01.320209 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003b11a0 after lost signal to thread 27701
W20260709 12:15:01.313403 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 449062us
W20260709 12:15:01.320415 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9f20 after lost signal to thread 27707
W20260709 12:15:01.322275 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800367400 after lost signal to thread 27708
W20260709 12:15:01.322650 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080035d8a0 after lost signal to thread 27710
W20260709 12:15:01.322942 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800317740 after lost signal to thread 27715
W20260709 12:15:01.323280 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080012cbe0 after lost signal to thread 27716
W20260709 12:15:01.323614 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800253bc0 after lost signal to thread 27718
W20260709 12:15:01.323953 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800278580 after lost signal to thread 27720
W20260709 12:15:01.324239 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080037b660 after lost signal to thread 27721
W20260709 12:15:01.324525 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800310000 after lost signal to thread 27722
W20260709 12:15:01.324818 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080036ff80 after lost signal to thread 27723
W20260709 12:15:01.326052 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5460 after lost signal to thread 27727
W20260709 12:15:01.326335 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0360 after lost signal to thread 27729
W20260709 12:15:01.326614 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080036bc80 after lost signal to thread 27744
W20260709 12:15:01.326977 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08002c9220 after lost signal to thread 27751
W20260709 12:15:01.327396 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8420 after lost signal to thread 27764
W20260709 12:15:01.328917 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003b3e60 after lost signal to thread 27774
I20260709 12:15:01.302713 27401 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:01.268079 27810 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:15:01.264864 27659 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:01.303750 27777 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:01.357816 27785 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.000s
I20260709 12:15:01.313208 27685 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.383315 27812 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:01.383884 27835 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:15:01.394130 27666 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:01.395161 27752 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:01.396674 27770 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:01.396622 27737 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.373457 27659 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:01.397985 27794 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.006s
I20260709 12:15:01.376392 27804 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:01.420188 27647 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.427035 27737 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.419092 27816 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:01.397989 27744 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.005s
I20260709 12:15:01.409317 27793 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.416610 27681 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.415123 27813 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:01.429067 27814 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.436333 27813 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:01.410272 27817 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.003s
I20260709 12:15:01.437166 27793 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.427439 27747 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.440623 27747 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.431973 27816 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:01.059654 23218 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:01.241119 23313 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:01.484298 23314 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.487066 23318 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.488009 23316 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:15:01.488701 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.004s	sys 0.004s
I20260709 12:15:01.492435 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2227) took 2698 ms. Trace:
I20260709 12:15:01.493129 23467 raft_consensus.cc:2410] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:15:01.493060 23469 rpcz_store.cc:276] 0709 12:14:58.793524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.649367 (+1855843us) service_pool.cc:224] Handling call
0709 12:15:00.649700 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:00.650052 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.650079 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:00.650612 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:15:00.942533 (+291921us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:00.942769 (+   236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:00.942913 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.491634 (+548721us) raft_consensus.cc:1685] finished
0709 12:15:01.491711 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.492119 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":788,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5648,"spinlock_wait_cycles":19072,"thread_start_us":227972,"threads_started":1,"wal-append.queue_time_us":514687}
W20260709 12:15:01.496038 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080036ca60 after lost signal to thread 27777
W20260709 12:15:01.496784 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003ddf80 after lost signal to thread 27782
W20260709 12:15:01.497108 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800261880 after lost signal to thread 27784
W20260709 12:15:01.497519 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800367540 after lost signal to thread 27785
W20260709 12:15:01.498028 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800196740 after lost signal to thread 27793
W20260709 12:15:01.498132 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.000s	sys 0.008s
W20260709 12:15:01.498504 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c9c80 after lost signal to thread 27794
W20260709 12:15:01.498710 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.000s	sys 0.008s
W20260709 12:15:01.499531 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 672905us
W20260709 12:15:01.501710 23462 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.002s	sys 0.011s
W20260709 12:15:01.502262 23462 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.002s	sys 0.011s
W20260709 12:15:01.489274 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.004s	sys 0.004s
I20260709 12:15:01.489238 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2351) took 2601 ms. Trace:
W20260709 12:15:01.501710 23465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.011s
I20260709 12:15:01.524034 23316 rpcz_store.cc:276] 0709 12:14:58.888140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.211141 (+2323001us) service_pool.cc:224] Handling call
0709 12:15:01.253778 (+ 42637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.488679 (+234901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.489127 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:01.544677 23465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.008s	sys 0.011s
I20260709 12:15:01.546577 24015 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 12:15:01.566129 23287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.006s	sys 0.009s
W20260709 12:15:01.569514 23287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.006s	sys 0.010s
I20260709 12:15:01.425274 27647 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.582885 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:01.602496 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2228) took 2808 ms. Trace:
I20260709 12:15:01.607009 24016 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:15:01.609254 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 257732us
I20260709 12:15:01.336128 27465 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.613970 27681 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:01.618268 23287 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.640s (SENT)
I20260709 12:15:01.613946 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:01.604091 23462 rpcz_store.cc:276] 0709 12:14:58.793792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.648686 (+1854894us) service_pool.cc:224] Handling call
0709 12:15:00.648990 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.649171 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:00.698205 (+ 49034us) log.cc:844] Serialized 14 byte log entry
0709 12:15:01.595067 (+896862us) log.cc:844] Serialized 14 byte log entry
0709 12:15:01.597490 (+  2423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.601776 (+  4286us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:01.601916 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.602039 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.602403 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51490,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":31580,"raft.run_cpu_time_us":389,"raft.run_wall_time_us":387,"thread_start_us":895469,"threads_started":2,"wal-append.queue_time_us":659401}
I20260709 12:15:01.622985 24008 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:15:01.623154 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 95137.1us
I20260709 12:15:01.625201 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:01.626729 27465 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.628103 27794 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.629407 27794 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.631223 23317 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:15:01.631939 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2365) took 1168 ms. Trace:
I20260709 12:15:01.632175 27794 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.632375 23317 rpcz_store.cc:276] 0709 12:15:00.464176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.630044 (+1165868us) service_pool.cc:224] Handling call
0709 12:15:01.630739 (+   695us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.631646 (+   907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.631862 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:01.633745 27793 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:01.634423 27793 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:01.636099 27793 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.345155 27401 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:01.644116 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1104) took 1007 ms. Trace:
I20260709 12:15:01.644763 23314 rpcz_store.cc:276] 0709 12:15:00.636203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.640410 (+1004207us) service_pool.cc:224] Handling call
0709 12:15:01.640914 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.641324 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.641361 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.641421 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:01.643775 (+  2354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.643849 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.644021 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:01.646315 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1086) took 3012 ms (client timeout 3021 ms). Trace:
I20260709 12:15:01.646449 23470 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:01.651973 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.652184 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2544) took 1310 ms. Trace:
I20260709 12:15:01.653228 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:01.653723 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 985) took 1382 ms (client timeout 1672 ms). Trace:
I20260709 12:15:01.654006 23319 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:01.655220 23388 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:01.655308 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.656381 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.663708 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.663959 27465 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:01.664870 23387 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:01.665529 27744 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:01.663725 27747 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:15:01.666023 27744 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:01.666529 27744 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.666565 27747 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:01.666793 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1883) took 1176 ms. Trace:
I20260709 12:15:01.667171 27747 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:01.667353 23391 rpcz_store.cc:276] 0709 12:15:00.489876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.608801 (+118925us) service_pool.cc:224] Handling call
0709 12:15:00.819288 (+210487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.858801 (+ 39513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.666341 (+807540us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002d1ed0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.666594 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483382016}
W20260709 12:15:01.658528 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44980us
I20260709 12:15:01.656104 27549 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.657924 27401 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:01.610469 27556 consensus_meta.cc:287] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.843s	user 0.008s	sys 0.008s
I20260709 12:15:01.660436 23464 rpcz_store.cc:276] 0709 12:15:00.342086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.522253 (+180167us) service_pool.cc:224] Handling call
0709 12:15:01.651784 (+1129531us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001874d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.651970 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249211136}
W20260709 12:15:01.661047 24009 rpcz_store.cc:269] 0709 12:15:00.270586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.353876 (+ 83290us) service_pool.cc:224] Handling call
0709 12:15:01.653209 (+1299333us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54001df454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.653447 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751226624}
W20260709 12:15:01.658754 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 337024us
I20260709 12:15:01.678202 23317 pending_rounds.cc:85] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 3
I20260709 12:15:01.683890 23394 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:01.684401 24007 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:15:01.684548 23392 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:01.659248 23318 rpcz_store.cc:269] 0709 12:14:58.634236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.251781 (+1617545us) service_pool.cc:224] Handling call
0709 12:15:01.646011 (+1394230us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000ec254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.646241 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622822144}
I20260709 12:15:01.641525 27794 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:01.686615 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.668855 27747 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.695779 23317 pending_rounds.cc:107] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 12:15:01.703624 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.700570 23317 raft_consensus.cc:2889] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:01.704975 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 897) took 1406 ms. Trace:
I20260709 12:15:01.705601 23463 rpcz_store.cc:276] 0709 12:15:00.298070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.621162 (+323092us) service_pool.cc:224] Handling call
0709 12:15:01.704491 (+1083329us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.704777 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380537728}
I20260709 12:15:01.698792 27556 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.707978 23218 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 8 to 10, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:01.709820 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.711054 23389 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:01.711630 23218 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:01.714078 27846 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:01.716892 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.718113 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.717350 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:01.716913 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.697716 27794 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:01.723659 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.724113 27846 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:01.724759 27661 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:01.725184 23314 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
W20260709 12:15:01.726312 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.726366 27681 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:01.728360 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:01.742377 23318 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:01.746049 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:01.747520 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:01.746026 23392 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:01.751250 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.752780 24013 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:01.734349 23314 pending_rounds.cc:85] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:15:01.753296 23314 pending_rounds.cc:107] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:15:01.753681 23314 raft_consensus.cc:2889] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:01.755929 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.733815 27794 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:01.759701 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.743458 27747 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:01.745975 27549 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:01.740361 27846 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:01.777101 27846 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:01.781311 27558 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:15:01.782389 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1097) took 2986 ms. Trace:
I20260709 12:15:01.782510 27465 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:01.765527 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1773) took 1023 ms. Trace:
I20260709 12:15:01.782934 27465 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:01.783068 23314 rpcz_store.cc:276] 0709 12:15:00.742091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.696953 (+954862us) service_pool.cc:224] Handling call
0709 12:15:01.724547 (+ 27594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.765180 (+ 40633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.765418 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:01.784674 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.784639 27465 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.785989 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.786448 27465 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:01.767448 27780 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:01.774720 23316 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:01.775188 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.776175 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.789839 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2360) took 1358 ms. Trace:
I20260709 12:15:01.782864 23320 rpcz_store.cc:276] 0709 12:14:58.795390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.713113 (+1917723us) service_pool.cc:224] Handling call
0709 12:15:00.713454 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:00.788867 (+ 75413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.788894 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:00.788935 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:00.788967 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.781752 (+992785us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:01.781840 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.782162 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172920704}
I20260709 12:15:01.795228 23313 rpcz_store.cc:276] 0709 12:15:00.431667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.484456 (+1052789us) service_pool.cc:224] Handling call
0709 12:15:01.485308 (+   852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.528285 (+ 42977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.528331 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:01.545972 (+ 17641us) log.cc:844] Serialized 26 byte log entry
0709 12:15:01.679431 (+133459us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:01.679467 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.679531 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.789469 (+109938us) raft_consensus.cc:1685] finished
0709 12:15:01.789489 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.789640 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48562,"raft.run_cpu_time_us":509,"raft.run_wall_time_us":49485,"thread_start_us":132898,"threads_started":1,"wal-append.queue_time_us":241081}
I20260709 12:15:01.804816 24009 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:01.811164 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:01.795935 27556 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:01.819336 23314 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:01.810163 24011 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 5.
I20260709 12:15:01.824471 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2362) took 1367 ms. Trace:
I20260709 12:15:01.825033 23316 rpcz_store.cc:276] 0709 12:15:00.456848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.569463 (+1112615us) service_pool.cc:224] Handling call
0709 12:15:01.573578 (+  4115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.791353 (+217775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.824157 (+ 32804us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400295254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:01.824375 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425229184}
W20260709 12:15:01.831215 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 83386.7us
I20260709 12:15:01.830327 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.844976 23312 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:01.846159 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.847021 23312 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.848028 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.848728 23312 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.842599 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.832764 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.851573 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.864117 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.869565 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1033) took 1015 ms. Trace:
I20260709 12:15:01.870041 24016 rpcz_store.cc:276] 0709 12:15:00.853683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.863265 (+  9582us) service_pool.cc:224] Handling call
0709 12:15:00.863675 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.607565 (+743890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.860686 (+253121us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54005ed654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:01.869488 (+  8802us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1707842944}
I20260709 12:15:01.871567 23396 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:01.872673 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1885) took 1315 ms. Trace:
I20260709 12:15:01.874261 27758 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:01.874589 27841 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:01.875379 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.873090 23390 rpcz_store.cc:276] 0709 12:15:00.506122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.618110 (+111988us) service_pool.cc:224] Handling call
0709 12:15:00.618425 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:00.630803 (+ 12378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:00.630831 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:00.775873 (+145042us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b1800126940. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a
0709 12:15:01.389930 (+614057us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b4800002dd8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:15:01.617231 (+227301us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:01.624737 (+  7506us) log.cc:844] Serialized 52 byte log entry
0709 12:15:01.756767 (+132030us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:01.758589 (+  1822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.761548 (+  2959us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.813625 (+ 52077us) raft_consensus.cc:1685] finished
0709 12:15:01.813757 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.821542 (+  7785us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11999,"raft.queue_time_us":18206,"raft.run_cpu_time_us":5154,"raft.run_wall_time_us":748133,"spinlock_wait_cycles":42703232,"thread_start_us":128347,"threads_started":1,"wal-append.queue_time_us":181355}
I20260709 12:15:01.875514 27758 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:01.878708 23318 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:01.879312 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1774) took 1137 ms. Trace:
I20260709 12:15:01.879599 23318 rpcz_store.cc:276] 0709 12:15:00.742331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.741424 (+999093us) service_pool.cc:224] Handling call
0709 12:15:01.742035 (+   611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.879067 (+137032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.879260 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:01.882382 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 154241us
I20260709 12:15:01.821898 27780 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:01.910527 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.911213 23316 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.884579 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:01.912019 23393 raft_consensus.cc:2410] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:01.913283 23392 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:01.913475 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.914341 23389 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.921054 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 759) took 1551 ms. Trace:
I20260709 12:15:01.921563 24013 rpcz_store.cc:276] 0709 12:15:00.369633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.408032 (+ 38399us) service_pool.cc:224] Handling call
0709 12:15:01.920793 (+1512761us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54005613d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.920988 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758960000}
I20260709 12:15:01.921870 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.923446 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1099) took 2090 ms. Trace:
I20260709 12:15:01.923826 23319 rpcz_store.cc:276] 0709 12:14:59.832968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.247119 (+1414151us) service_pool.cc:224] Handling call
0709 12:15:01.653619 (+406500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.678926 (+ 25307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.923212 (+244286us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b5400389d50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.923370 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":933806464}
I20260709 12:15:01.924681 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:01.925846 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.926405 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2352) took 1503 ms. Trace:
I20260709 12:15:01.926800 23321 rpcz_store.cc:276] 0709 12:15:00.423376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.249637 (+826261us) service_pool.cc:224] Handling call
0709 12:15:01.250030 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:01.250496 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.250537 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:01.256368 (+  5831us) log.cc:844] Serialized 26 byte log entry
0709 12:15:01.925698 (+669330us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b5400475020. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a95eb6b8 00007ff0a967492b 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c
0709 12:15:01.925789 (+    91us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:01.925856 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.925956 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:01.925998 (+    42us) raft_consensus.cc:1685] finished
0709 12:15:01.926017 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.926230 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2841,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":3909,"spinlock_wait_cycles":41389440,"thread_start_us":120577,"threads_started":1,"wal-append.queue_time_us":133298}
I20260709 12:15:01.927942 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:01.928531 23468 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.009s
I20260709 12:15:01.928979 23468 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:01.930814 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2242) took 1080 ms. Trace:
I20260709 12:15:01.931104 23468 rpcz_store.cc:276] 0709 12:15:00.849896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.976955 (+127059us) service_pool.cc:224] Handling call
0709 12:15:01.930574 (+953619us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54001879d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.930699 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749292800}
I20260709 12:15:01.880498 27758 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:01.928907 23396 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:01.940091 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:01.949384 23388 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:01.952409 27758 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.875414 27841 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.953958 27758 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:01.956079 27841 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:01.957909 27853 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:15:01.962375 23390 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:15:01.963846 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.964648 23315 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:15:01.965871 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.966475 27625 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:01.966745 23389 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
W20260709 12:15:01.965978 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 48114.2us
I20260709 12:15:01.967036 27625 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.969619 27625 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:01.971855 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 892) took 3038 ms. Trace:
I20260709 12:15:01.972226 23461 rpcz_store.cc:276] 0709 12:14:58.933464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:00.721110 (+1787646us) service_pool.cc:224] Handling call
0709 12:15:01.664252 (+943142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:01.664703 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.664740 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:01.664783 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:01.664811 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:01.664881 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:01.971569 (+306688us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54001874d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:01.971678 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2156127488}
W20260709 12:15:01.974845 27859 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.192s	user 0.002s	sys 0.000s
I20260709 12:15:01.976488 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.977363 23388 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
W20260709 12:15:01.977979 27863 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.155s	user 0.002s	sys 0.001s
I20260709 12:15:01.978574 23319 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:01.982062 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.983217 23316 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:01.984388 27780 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:01.984555 23316 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:15:01.985061 23316 pending_rounds.cc:85] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:15:01.986585 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.990648 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:01.992035 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:01.992798 23389 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:01.996485 27558 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:15:01.999383 27864 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:15:02.000305 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:02.002494 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:02.004693 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:02.005975 24013 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:02.009131 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.010085 23392 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.008991 27816 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:02.010623 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:02.012225 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.014175 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:02.017035 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:02.019111 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 68326.2us
I20260709 12:15:02.020148 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:01.869083 27744 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:01.985287 23316 pending_rounds.cc:107] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:15:02.025086 23316 raft_consensus.cc:2889] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:01.996861 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.021667 27549 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:15:02.038695 24011 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:02.027887 27681 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:02.046136 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.048835 27685 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:02.050498 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:02.047360 23318 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.055804 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:02.056761 27625 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:02.057155 27625 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.057618 27625 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:02.044275 27558 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:02.070164 27647 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:02.070760 27647 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:02.070344 27625 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.041661 27816 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:02.074501 27758 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:02.074581 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.074971 27758 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:02.075860 23461 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.075968 27758 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.077934 27758 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.079982 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.081151 23387 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:02.081295 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.081786 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.082422 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.083760 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.084462 23388 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:15:02.084349 27794 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.088845 27625 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:02.090615 27744 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:02.098393 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58138.3us
I20260709 12:15:02.099294 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:02.100378 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.040634 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:02.101483 23392 raft_consensus.cc:606] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.064834 27846 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.084374 23393 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:02.101498 27637 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:02.090767 27681 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.125970 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.099985 27647 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.112623 23314 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:02.120399 23318 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
W20260709 12:15:02.122710 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77795.1us
I20260709 12:15:02.124396 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.125232 27685 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:02.125929 27846 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.127588 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:02.128331 27637 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:02.134562 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:02.145475 23315 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:02.139608 23394 raft_consensus.cc:2410] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:15:02.140223 23393 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.141788 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.142781 23312 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:02.147408 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.147670 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.148412 23396 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.150281 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.137539 23461 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:02.151611 27681 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.156757 24015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:02.157797 24015 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:02.160100 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1770) took 1445 ms. Trace:
I20260709 12:15:02.159970 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:02.160769 23317 rpcz_store.cc:276] 0709 12:15:00.714625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:01.643281 (+928656us) service_pool.cc:224] Handling call
0709 12:15:01.677736 (+ 34455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:01.705310 (+ 27574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:01.705358 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:01.706019 (+   661us) log.cc:844] Serialized 52 byte log entry
0709 12:15:02.027080 (+321061us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:15:02.027118 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:02.027249 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:02.159843 (+132594us) raft_consensus.cc:1685] finished
0709 12:15:02.159864 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:02.160034 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1565,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":172,"thread_start_us":320611,"threads_started":1,"wal-append.queue_time_us":451404}
I20260709 12:15:02.161899 23316 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:15:02.163055 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 166411us
I20260709 12:15:02.163076 27744 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:02.166857 27744 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:02.151160 23388 raft_consensus.cc:2376] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13. Re-sending same reply.
I20260709 12:15:02.157207 27681 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:02.170590 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:02.173309 27549 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:02.173797 27549 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:02.174203 27549 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:02.181804 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:02.184590 27758 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:02.185279 27758 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:02.174163 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:02.192859 27647 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.194345 27465 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:02.195751 27558 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:02.196225 27558 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:02.197448 27558 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.198874 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:02.198552 27637 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.168546 27744 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.210876 23314 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:02.230991 27552 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:15:02.235915 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 84983.7us
I20260709 12:15:02.248310 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:02.249136 27780 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:02.255007 27780 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.258813 27782 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:02.259294 27782 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.259657 27782 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.249972 27758 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.190896 27848 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:15:02.253240 27794 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.261839 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:02.185602 27549 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.260349 27465 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:15:02.262357 27780 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.266932 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:02.267412 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.268362 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:02.276084 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 148627us
I20260709 12:15:02.283888 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:02.293227 27879 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 12:15:02.279611 27625 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:02.285887 27780 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:15:02.311600 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52720.1us
W20260709 12:15:02.313131 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 261603us
W20260709 12:15:02.337921 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 73970us
I20260709 12:15:02.339335 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:02.342643 27549 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.306207 27647 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:15:02.311728 27625 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:02.359035 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.359411 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:02.363579 23982 leader_election.cc:336] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.614s (SENT)
I20260709 12:15:02.364352 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:02.346659 27780 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:02.367669 27780 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.369238 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:02.370112 27780 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:02.370352 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:02.370518 27558 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:02.372042 27637 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:02.372496 27637 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.372893 27637 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.374811 27846 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:02.375866 27846 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:02.375900 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:02.379362 27556 raft_consensus.cc:993] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:02.380584 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:02.381650 27549 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:02.382412 27549 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:02.383334 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:02.383533 27549 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.384677 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:02.385936 27401 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:15:02.386343 27401 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.386721 27401 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:02.389969 27586 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:02.390445 27586 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:02.390851 27586 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.393987 27817 raft_consensus.cc:2764] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:15:02.394925 27817 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:02.381755 27813 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:02.394829 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:02.396384 27884 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:02.396863 27552 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:02.397316 27552 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:02.399055 27552 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.399719 27625 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.400981 27552 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.403318 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:02.377125 27846 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.428453 27758 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:02.428572 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65098.8us
I20260709 12:15:02.429942 27841 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:02.431414 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:02.434082 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:02.434751 23470 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:02.422377 27556 raft_consensus.cc:1081] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:02.449358 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:02.453291 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:15:02.454094 23320 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:02.454511 23320 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:02.459859 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 100768us
I20260709 12:15:02.470261 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:02.487425 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 85320.7us
I20260709 12:15:02.448096 27782 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:02.493916 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 111505us
W20260709 12:15:02.494099 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46495us
I20260709 12:15:02.472877 27794 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:02.483357 27813 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:02.510443 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.218s (SENT) [suppressed 31 similar messages]
I20260709 12:15:02.516309 23465 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:02.497498 27401 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.519861 27401 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.510007 27637 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.526592 27782 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:02.532743 23320 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.535789 27794 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:02.536633 27890 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:02.538151 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.539949 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.540726 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.576630 24014 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:02.576903 23320 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.541226 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.541464 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.578081 24013 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.542459 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:02.542563 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.542654 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.582326 23395 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:15:02.586225 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 57886.8us
I20260709 12:15:02.593312 24011 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:15:02.606381 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64024.8us
W20260709 12:15:02.606766 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65863.4us
I20260709 12:15:02.581768 27794 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.611326 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.616824 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.622537 24011 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:02.622803 27637 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:02.623772 27893 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 12:15:02.623909 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:02.624573 27892 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 12:15:02.625255 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:02.625838 23465 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.629685 27465 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:02.629997 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:02.630616 23468 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:02.631328 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 85737.4us
I20260709 12:15:02.624543 23470 raft_consensus.cc:2376] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9. Re-sending same reply.
I20260709 12:15:02.638170 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:02.638751 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:02.641450 23314 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:02.630926 27465 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:02.646277 23316 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:02.631609 27817 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:02.648746 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:02.654379 23470 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:02.639411 27793 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:02.658922 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.614626 27401 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:02.668319 27586 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.663893 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:02.673048 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.671346 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.643810 27465 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:02.677280 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 90057.4us
I20260709 12:15:02.657567 27817 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:02.692555 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:02.692795 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.648712 27890 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:02.695487 23394 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.670672 27586 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:02.708673 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.708453 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:02.709271 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.710493 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:02.713582 23470 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:02.719272 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:02.696302 27817 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:02.722205 27908 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:15:02.723227 23389 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:02.719825 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:02.737423 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:02.743219 23465 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:15:02.749034 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 107635us
I20260709 12:15:02.754449 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:02.754730 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:02.755754 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:02.756682 24013 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:02.759413 23468 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:02.763273 27909 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:15:02.765156 27900 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.765738 27900 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.768047 27900 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:02.722958 27890 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:02.774286 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65940.8us
I20260709 12:15:02.745172 27637 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:02.725865 27841 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:15:02.807965 24008 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:02.818608 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:02.825107 27904 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:15:02.827267 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:02.827993 23466 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:02.829347 27914 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.829908 27914 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.832067 27914 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:02.819267 27794 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:02.833196 27906 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:15:02.834892 27794 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:02.839385 27902 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:15:02.797960 27465 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.850634 27465 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.853256 24011 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:02.853766 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:02.854902 27896 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:02.855289 27896 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:02.855352 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.855731 27896 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.856750 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.857618 27900 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:02.858345 27900 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:15:02.858522 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44548.1us
I20260709 12:15:02.859596 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:02.860431 27681 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:02.860890 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:02.860996 27681 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:02.862840 27681 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.861605 27549 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.865868 27549 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:02.866303 27549 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.873756 27637 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:02.874275 27637 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:02.874662 27637 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:02.894129 23392 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:02.873764 27647 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:02.901278 27896 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.904347 27914 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:02.906266 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:02.910655 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.914177 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.915386 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.916080 27794 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:02.916499 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.916765 27794 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:02.906003 27780 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:15:02.928664 27900 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:02.930658 27794 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:02.931213 27681 raft_consensus.cc:2764] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:15:02.910465 27647 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.954864 27647 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.955745 27637 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:02.927712 27817 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:02.951947 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:02.963707 27888 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.273s	user 0.002s	sys 0.000s
I20260709 12:15:02.971210 27817 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:02.971087 27793 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:02.971833 27793 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.966230 27647 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:02.979168 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:02.980485 27647 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:02.984223 27897 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:02.988801 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:02.991179 27681 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:02.985230 27897 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:02.993777 27897 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:02.998948 27637 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:02.984575 27647 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:02.989540 27896 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:02.986317 27549 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:03.023684 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46315.1us
I20260709 12:15:03.027060 27549 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.030622 27647 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.033221 27647 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.045039 27793 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.047508 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.037854 23394 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:03.048400 27647 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:03.048894 27647 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:03.049667 27647 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.047631 27793 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:03.033603 27931 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
I20260709 12:15:03.058401 27549 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:15:03.071259 27647 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.076157 27647 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:03.077675 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 298381us
W20260709 12:15:03.084277 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.089758 27549 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:03.090163 23465 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:03.091327 27549 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:03.095889 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.101383 23317 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:03.111441 27921 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:03.112618 27586 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:03.113024 27586 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:03.113946 27586 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.115057 27586 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.102702 27647 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.117659 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.119532 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:03.121639 27465 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.122324 23313 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:03.123366 27465 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:03.123955 27465 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.125249 23320 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:03.125574 27884 raft_consensus.cc:993] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:15:03.126127 27884 raft_consensus.cc:1081] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:15:03.137696 23315 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:03.138995 23317 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:03.124878 27465 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.148056 27794 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:03.148650 23312 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:03.150271 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45259.9us
I20260709 12:15:03.133853 27897 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.157770 27921 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:03.159186 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:03.160107 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.161369 27794 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:03.161808 27794 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.162878 27794 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.163444 27625 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:03.164208 27794 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.164938 27625 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:03.167037 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:03.167817 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48399.5us
I20260709 12:15:03.166035 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.169706 23315 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:03.170688 27647 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.184623 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.171628 27919 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:15:03.173573 27816 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.189049 27897 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.189178 23313 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:03.176378 27921 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:15:03.190662 27921 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:15:03.191051 27921 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:03.193974 27900 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:15:03.198089 27900 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:03.198490 27900 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.199362 27900 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.200547 27900 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.188750 27816 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.205109 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.205199 27901 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:03.206229 23320 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:15:03.206435 27901 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:03.206817 27901 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.207645 27901 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.214507 27777 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:03.214665 27901 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.214941 27777 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:03.215438 27729 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:03.215929 27777 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.216555 27729 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:03.217001 27729 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:03.218104 27729 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.219640 27729 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:03.194630 27549 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:03.223464 27549 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:03.223857 27549 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.226346 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:03.230465 23219 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 4 to 10, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:03.231904 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56868.6us
I20260709 12:15:03.233330 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.232848 23219 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:03.235010 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.234776 23219 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:03.176668 27934 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:03.240834 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.241096 27933 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:03.217180 27777 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.269687 23461 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:03.270861 27681 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:03.270941 27897 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.271374 27681 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:03.271860 27681 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:03.271652 27897 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.251636 27933 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:03.273758 27933 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:03.275559 27897 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.276491 23219 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:03.276655 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:03.281477 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.291358 23313 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:03.296204 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:03.297147 27897 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.297609 27897 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.297964 27897 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.298581 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:03.248868 27934 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.241173 27816 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.303305 23465 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:03.316890 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:03.318023 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83069.9us
W20260709 12:15:03.319624 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 78877.9us
I20260709 12:15:03.290351 27813 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.320976 23466 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:03.322486 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.323621 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:03.328225 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:03.321311 27647 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:15:03.303107 27934 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:03.321570 27813 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:03.237753 27921 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:03.340003 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45423.6us
I20260709 12:15:03.342159 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.342662 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.348024 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:03.349577 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:03.350200 24016 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:03.354200 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.344453 23219 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 6 to 8, leader changed from <none> to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 8 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:03.344427 27813 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:03.371863 23218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 343) took 20529 ms (client timeout 15000 ms). Trace:
I20260709 12:15:03.346184 27637 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:03.344205 27586 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:03.374343 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 55244.3us
I20260709 12:15:03.378676 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.386231 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.386893 24009 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:03.386921 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.388243 24007 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:03.389544 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.392091 27817 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.396978 24009 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:03.403388 27897 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.406132 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:03.406966 24015 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:03.406858 27897 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.407827 27814 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.408329 27814 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.413893 24012 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:03.415174 27921 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:03.389631 27637 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:03.376029 27586 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:03.425894 27586 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:03.426502 27939 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:15:03.427827 27909 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.428350 27909 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.430575 27909 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.434441 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.435128 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:03.435767 23320 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.436122 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.436990 27899 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.437598 27914 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:15:03.437984 27899 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.438755 23315 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:03.437789 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.443208 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 22fc2096e78f47158bbb4bae3a474e2c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:43504
W20260709 12:15:03.444824 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da1e2fd157894634bf92d6c82c06ec64 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:03.446012 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:03.446632 23465 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:03.449208 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:03.447407 23468 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:15:03.448362 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:03.409988 27814 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.459204 23466 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:15:03.457208 23470 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:03.463828 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:03.467021 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.467872 23393 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:03.468528 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:03.474054 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55527.9us
I20260709 12:15:03.480123 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.473308 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:03.483457 23320 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:03.484485 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.485318 23393 raft_consensus.cc:2410] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:03.484637 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:03.488670 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.489904 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.492079 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.494264 23390 raft_consensus.cc:2368] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:15:03.496381 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.491423 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.499639 23395 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:03.490748 23388 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:03.504511 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.506425 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.508703 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:03.509123 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.509213 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.511548 23391 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:15:03.509884 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.457698 27817 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:15:03.510422 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.457862 27794 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:15:03.448257 27681 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.520624 23395 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:03.456282 27933 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:03.522954 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 147183us
I20260709 12:15:03.517879 27899 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.515544 27549 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.524736 27899 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:03.526338 27899 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:03.526546 27549 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:03.527179 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.529680 23319 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:03.532548 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.533509 23392 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:03.534114 23391 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:03.534265 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.535049 23392 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:03.535431 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.534114 23387 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:03.536372 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.489455 27637 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.525169 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.535782 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.542935 23392 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:03.541031 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.511534 27897 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:15:03.373842 23218 rpcz_store.cc:269] 0709 12:14:42.842054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:42.862127 (+ 20073us) service_pool.cc:224] Handling call
0709 12:15:03.371723 (+20509596us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:02.084701 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:02.085355 (+   654us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:02.085403 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:15:02.085469 (+    66us) tablet.cc:662] Decoding operations
0709 12:15:02.086534 (+  1065us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:02.086643 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:02.086670 (+    27us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:15:02.087539 (+   869us) tablet.cc:701] Row locks acquired
0709 12:15:02.087587 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:15:02.087767 (+   180us) write_op.cc:270] Start()
0709 12:15:02.087913 (+   146us) write_op.cc:276] Timestamp: P: 1783599302087701 usec, L: 0
0709 12:15:02.087950 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:15:02.143373 (+ 55423us) log.cc:844] Serialized 3693 byte log entry
0709 12:15:02.511194 (+367821us) op_driver.cc:464] REPLICATION: finished
0709 12:15:02.745821 (+234627us) write_op.cc:301] APPLY: starting
0709 12:15:02.765571 (+ 19750us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:02.847398 (+ 81827us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:02.847542 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:02.916049 (+ 68507us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:02.916493 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:15:02.916585 (+    92us) write_op.cc:312] APPLY: finished
0709 12:15:02.942751 (+ 26166us) log.cc:844] Serialized 89 byte log entry
0709 12:15:02.965903 (+ 23152us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:02.974219 (+  8316us) write_op.cc:454] Released schema lock
0709 12:15:02.979378 (+  5159us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:03.006447 (+ 27069us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:03.007520 (+  1073us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46338,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":165382,"apply.run_cpu_time_us":18046,"apply.run_wall_time_us":274464,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41783,"num_ops":9,"prepare.queue_time_us":198008,"prepare.run_cpu_time_us":8696,"prepare.run_wall_time_us":184334,"raft.queue_time_us":187218,"raft.run_cpu_time_us":45467,"raft.run_wall_time_us":146656,"replication_time_us":410474,"thread_start_us":226418,"threads_started":4,"wal-append.queue_time_us":82629}]]}
I20260709 12:15:03.545976 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.548810 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.549394 23395 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.551262 23396 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:15:03.551604 23396 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 8
I20260709 12:15:03.551836 23396 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:15:03.552067 23396 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:03.554049 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:03.559558 23392 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:03.559932 23392 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.561915 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:03.564757 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.566074 23393 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:15:03.573361 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:03.573784 23393 pending_rounds.cc:85] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 8
I20260709 12:15:03.566135 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.574107 23393 pending_rounds.cc:107] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:15:03.574079 23461 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:15:03.574795 23461 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:03.575121 23461 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.576040 23393 raft_consensus.cc:2889] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:03.578727 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:03.579250 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.579401 24012 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:03.580268 27552 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.580498 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.580998 27552 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:03.581548 27911 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.581771 27552 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.582751 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.583365 24008 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:03.581817 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.590633 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:03.585538 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60486us
I20260709 12:15:03.594441 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:03.596889 24015 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:03.599155 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 75072.1us
I20260709 12:15:03.561576 27549 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:03.595952 27919 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.567302 27897 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.544459 27816 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:15:03.604388 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.548491 27841 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:15:03.606444 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.607950 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:03.606614 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:03.583107 27896 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:03.595964 27794 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:03.608274 27884 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:15:03.616518 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:03.625099 23434 leader_election.cc:336] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.541s (SENT)
I20260709 12:15:03.590687 27934 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.629693 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:03.607237 27901 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:03.585338 27899 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.649726 27900 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:03.650332 27816 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:03.646692 27944 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:15:03.689599 27813 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.692083 27813 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.692245 27934 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:03.630749 27896 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:03.615469 27814 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:03.693390 27896 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.693783 27814 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:03.694420 27814 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:03.694746 27896 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.696451 27896 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:03.667430 27817 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:03.702114 27817 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:03.651417 23319 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:03.651311 23461 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:03.665175 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:03.705325 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73446.8us
I20260709 12:15:03.703073 27817 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.703910 27816 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:03.713171 27816 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:03.714931 23463 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:03.716814 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 115050us
I20260709 12:15:03.717728 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.718163 23317 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:03.718580 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:03.719022 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.719180 27552 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:03.719556 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 111668us
I20260709 12:15:03.720002 27552 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:03.720418 23982 leader_election.cc:400] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
I20260709 12:15:03.729988 23982 leader_election.cc:403] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:15:03.682605 23392 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.730563 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:03.686136 27897 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:03.687490 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:03.720988 27900 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.722206 23312 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:03.717172 23313 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:03.723174 27681 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.726862 27847 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:03.726999 27586 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.668205 24015 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:03.730940 23392 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:03.657320 27549 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:03.706378 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:03.738054 23389 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:03.683826 27465 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:03.744812 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.747349 23312 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:03.746073 23395 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:03.748639 23461 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:03.750710 23468 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:03.733489 27909 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:03.754052 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.759702 27903 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:03.760134 27903 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:03.761094 27903 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.762586 27903 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:03.750578 27890 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:03.764209 27890 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:03.731339 27782 consensus_queue.cc:1175] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "54640ab8abeb4c949426d66c7e5aed87" responder_term: 16 status { last_received { term: 16 index: 13 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:03.766120 27586 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:03.766386 27890 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.750185 27647 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:03.767731 27647 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:03.768137 27647 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:03.768787 23392 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:03.769893 27900 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:03.775635 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 105084us
I20260709 12:15:03.775933 23313 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:03.778739 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:03.785342 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65109.2us
I20260709 12:15:03.660347 27901 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.786053 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:03.786921 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 56440.3us
I20260709 12:15:03.788300 23463 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:03.781252 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 22fc2096e78f47158bbb4bae3a474e2c (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:15:03.754069 27549 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:03.821647 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.822986 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:03.824887 23206 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43705us
I20260709 12:15:03.757496 27909 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.825029 23312 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:03.817128 27465 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.788674 27586 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:03.830953 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43939.5us
I20260709 12:15:03.789376 27847 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12.
W20260709 12:15:03.834573 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57697.8us
I20260709 12:15:03.835490 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.835222 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.785235 27841 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:15:03.788836 27552 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:15:03.826864 27900 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:03.867089 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:03.873579 23313 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:15:03.829417 27744 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:03.822636 27890 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:15:03.856480 27933 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:03.879439 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46668.6us
W20260709 12:15:03.883715 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48323.2us
I20260709 12:15:03.885124 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.894726 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.895879 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:03.896586 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 148937us
I20260709 12:15:03.896953 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.898623 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.900563 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:03.902207 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.888856 27793 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:03.898777 27777 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.633s	user 0.006s	sys 0.006s
I20260709 12:15:03.925463 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.878324 27900 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.904022 27782 raft_consensus.cc:2764] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:15:03.931807 23434 leader_election.cc:336] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.754s (SENT)
I20260709 12:15:03.914920 27890 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:03.940543 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.970777 27465 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:03.979707 27465 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:03.982987 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.985231 24013 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:03.929481 27955 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:15:03.988147 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47594.5us
W20260709 12:15:03.989537 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89036.4us
I20260709 12:15:03.893641 27909 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:03.990495 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:03.955121 27729 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:03.991673 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 90866.2us
I20260709 12:15:03.976157 27899 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:03.970088 27841 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:03.915328 27549 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:03.997098 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:03.980372 27793 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:15:03.969420 27814 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.008796 27900 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.009559 27957 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:04.010098 27957 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.011433 27816 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.012156 27957 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:03.996383 27901 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.017819 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.019183 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.020752 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.020957 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.022126 23395 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:04.022778 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.023420 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.024041 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:04.024936 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:04.025169 23319 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:04.026170 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.027665 23392 raft_consensus.cc:2410] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:15:04.027648 27919 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:04.028232 27919 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.028606 27919 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.029637 27777 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:04.019733 23313 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:15:04.031406 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.029958 27744 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:04.035713 23466 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.035997 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:03.997762 27841 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.037151 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:04.005421 27955 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:04.035006 27744 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:04.041018 23470 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.036914 27958 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:15:04.042258 27958 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:04.041917 27744 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.042650 27958 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:04.045225 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.048069 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.048856 23394 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:04.026484 27903 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:15:04.049249 23394 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:04.055902 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.056526 23393 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.057307 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.057832 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.057845 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.058552 23470 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 6: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:04.036008 27816 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.059134 23391 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:15:04.061136 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:04.044186 27841 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.037465 27647 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.064357 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:04.017719 27465 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.061928 23394 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.071396 23391 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:04.071388 23394 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.073122 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.075439 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.076043 23390 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:04.076799 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.079074 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.080092 27899 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:04.080487 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.059962 27934 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.080760 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.081923 23315 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:04.082922 23461 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:04.084306 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.062033 27955 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:04.080500 27814 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.088114 23317 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:04.088814 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.094338 23470 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
W20260709 12:15:04.094842 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.096195 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.096810 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.096907 24008 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:04.096020 27914 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:15:04.099949 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.100085 27813 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:04.100479 27586 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.100674 27914 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:04.101143 27914 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:04.106643 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.080525 27899 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.098650 24016 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:04.111207 23389 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:04.083838 27782 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:04.086150 27817 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:04.085985 27890 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:15:04.084286 27900 consensus_queue.cc:1175] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2778b449761a46708f2f643b1b5b5854" responder_term: 11 status { last_received { term: 11 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 11 index: 7 } }
I20260709 12:15:04.097713 27465 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.131134 23315 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:15:04.140090 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.140910 23312 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:04.137812 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.074206 27953 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:15:04.118789 27899 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.147125 27813 consensus_queue.cc:1243] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:04.125831 27817 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.088637 27934 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.129858 27890 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:04.154666 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.155951 23394 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:04.161073 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.162213 23312 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.088924 27962 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:15:04.060077 27958 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.161113 23315 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:04.165606 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:04.167549 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60987.4us
I20260709 12:15:04.170811 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.172701 27959 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:15:04.175907 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.180819 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:04.181810 23434 leader_election.cc:336] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.814s (SENT)
I20260709 12:15:04.181955 23468 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:04.182569 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:04.184127 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.151810 27814 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.133253 27901 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:15:04.189793 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.197203 23466 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:04.206267 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.163709 27890 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:04.173270 27959 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:04.174463 27953 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:04.160169 27817 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.207641 23394 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:04.202044 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.209744 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.203563 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.210772 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.206521 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.216710 23463 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:04.217537 24009 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:04.212442 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.214747 23388 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:15:04.220569 23389 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:04.221971 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:04.215631 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.185336 27793 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:15:04.213490 27953 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:04.232311 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 143376us
I20260709 12:15:04.226507 27958 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:04.239486 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:04.237632 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118039us
I20260709 12:15:04.249632 27968 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.206218 27814 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.253764 24012 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:04.258322 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:04.262398 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80647.3us
I20260709 12:15:04.264120 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:04.280207 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46929us
I20260709 12:15:04.322345 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.323129 23392 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:04.259816 27647 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.337047 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.346422 23388 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:04.350664 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.351048 23393 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:04.352613 23390 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:15:04.355751 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.358191 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:04.359390 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 95321.7us
I20260709 12:15:04.364883 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:04.359028 27879 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:15:04.353597 27968 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:04.369773 23388 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:04.373463 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.359604 27637 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:04.376276 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.366319 27933 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:04.387815 23463 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:04.389540 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:04.395242 27911 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:04.395916 27911 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:04.411680 27968 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:04.421684 27972 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:15:04.435725 27973 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:15:04.438063 27970 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:15:04.445510 27979 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:04.446421 27979 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:04.451429 27967 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
W20260709 12:15:04.452157 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:15:04.454391 27919 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.383455 27934 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.490115 27971 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:04.490677 27971 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.491025 27971 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.492544 27793 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:04.493104 27793 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:04.493681 27958 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.494026 27793 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.494369 27958 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.500553 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:04.505858 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:04.507148 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 142776us
I20260709 12:15:04.507613 24008 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:04.511823 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.515038 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.515753 23313 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:04.517706 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.524765 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.497210 27899 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.497071 27958 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.529661 23465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43504
I20260709 12:15:04.531734 23465 raft_consensus.cc:606] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Received request to transfer leadership
I20260709 12:15:04.532095 23465 raft_consensus.cc:614] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:04.542506 24015 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:04.546525 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.547457 27955 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:04.547560 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:04.548628 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.502933 27794 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:04.555029 23312 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:04.560709 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.562758 23312 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:04.563623 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 198818us
I20260709 12:15:04.564730 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:04.530722 27933 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:04.569532 24016 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:04.570140 27549 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:04.570933 23388 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:15:04.576773 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.578161 23388 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:04.579247 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:04.552417 27814 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.580506 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:04.581322 23387 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:04.562767 27794 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.584606 27976 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:04.575804 27549 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.558393 27647 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:15:04.596249 27919 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.596380 27980 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:04.577587 27968 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:04.597970 27549 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.603238 27586 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:15:04.594530 27981 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:04.617683 27899 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:04.615620 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 393706us
I20260709 12:15:04.613296 27793 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:04.626471 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.610427 27980 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:04.629645 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48743.3us
I20260709 12:15:04.632121 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.340613 27953 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.397315 27911 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.637602 24016 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:04.637363 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89873.6us
I20260709 12:15:04.667299 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.447629 27979 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.641882 27984 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.641743 27975 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:15:04.641300 27959 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.646566 27985 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:04.656450 27744 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.621235 27976 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:04.665521 27794 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:04.666555 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 87450.3us
I20260709 12:15:04.614537 27890 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.676178 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:04.679435 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.639478 27901 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.691965 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.672986 27991 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.672622 27968 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:04.683954 27933 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:04.672241 27984 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.680650 27647 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:04.688575 27911 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:04.703347 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.636296 27793 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.695850 27586 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:15:04.709299 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.709717 27993 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:04.710038 24016 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:04.710104 27993 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:04.711164 27793 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.710971 27993 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.712190 27984 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.712273 27993 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.713985 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.719568 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:04.734128 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.734930 23470 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
W20260709 12:15:04.736116 27989 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:15:04.695750 27901 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.738749 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.738931 23313 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:15:04.738893 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:04.739948 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47161.7us
I20260709 12:15:04.741398 24016 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:04.717407 27975 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:04.741787 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.741686 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:04.747294 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.751919 24008 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:04.712761 27647 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.727692 27884 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:04.799098 27884 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:04.799549 27884 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:04.702759 27991 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.803570 27991 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.754856 27961 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:04.804198 27961 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:04.805119 27961 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.806674 27961 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.754606 27986 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:15:04.701920 27981 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 12:15:04.763113 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.737360 27980 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.763684 27971 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.765388 27647 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.766786 27890 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:04.750892 27817 consensus_meta.cc:287] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.002s
I20260709 12:15:04.742393 27586 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:04.744939 27793 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:04.769938 27997 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:15:04.775933 27841 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.005s
I20260709 12:15:04.747591 27976 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.747632 27813 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:04.790681 27914 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.792299 27909 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:04.795085 27992 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:04.707293 27968 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.797703 27681 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:04.798517 27901 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.811836 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.846482 23468 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:04.812893 27933 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:04.815712 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 124712us
W20260709 12:15:04.820001 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 78374.2us
I20260709 12:15:04.824533 23394 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:04.825537 27971 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:04.828527 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.837842 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:04.839212 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:04.850812 23461 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.850919 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:04.839210 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 97533.4us
I20260709 12:15:04.848369 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.838119 27841 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.830893 27793 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.838449 27813 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:04.857993 27793 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:04.859606 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:04.865213 23396 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:04.870437 24011 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:04.831142 27817 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.874140 24012 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:04.834261 27586 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.874152 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.872085 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.877795 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:04.879554 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.879070 24012 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:15:04.856796 27901 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:04.854482 27681 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:04.882452 24013 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:04.879390 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:04.863852 27729 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.898113 27729 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.900388 27729 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:04.904323 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.864152 27813 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.906450 23467 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:04.895045 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.895550 24009 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:04.908641 23391 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:04.881168 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:04.910205 23469 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.910715 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.911916 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.912712 23464 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:04.847052 27992 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.915380 28002 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:15:04.916498 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.917158 27995 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:04.919255 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.922173 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.922458 27979 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:04.923408 27979 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:04.923847 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:04.924441 23467 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:04.894378 27933 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:04.926926 23465 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.922869 23395 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:04.874785 27914 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.930063 23470 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:04.937275 23461 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:04.925333 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.945864 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.947276 23468 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:04.938135 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:04.953485 23395 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:04.922397 28009 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:04.961871 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 80787us
I20260709 12:15:04.944110 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:04.962834 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:04.961856 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:04.966536 23389 raft_consensus.cc:2376] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12. Re-sending same reply.
I20260709 12:15:04.948931 27901 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:04.987291 23393 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:04.987862 23467 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:15:04.988629 23461 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:15:04.991039 23464 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:15:04.994554 24016 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:15:04.996877 23393 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:05.001108 23465 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:05.030747 23468 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:04.969755 28007 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:15:05.031867 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46617.7us
I20260709 12:15:05.029212 23464 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:05.030172 23391 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:15:05.034269 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:05.032054 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:05.033710 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:05.043839 24009 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:04.997534 27901 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:04.990212 27890 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:05.062845 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:05.078896 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.072491 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:04.981648 27729 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.087587 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.087914 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:05.087913 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.088558 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.089978 23314 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:05.090109 24014 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:05.093184 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.000058 27777 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:15:05.103178 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.103161 23317 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:05.107034 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.117413 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:04.992905 27921 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:15:05.078013 28007 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:05.139806 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:05.140707 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:05.101343 27637 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:05.151779 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.153522 27958 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:05.154258 27958 consensus_queue.cc:1243] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:05.151350 27647 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:15:05.152458 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:05.149760 28015 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:05.155949 27956 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:05.155506 28019 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.156786 28019 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.157928 27586 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.160154 27884 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:05.161563 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.162005 28007 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.165146 23470 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:05.060150 27681 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.106642 27890 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:05.166172 23389 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
W20260709 12:15:05.166742 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.166653 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 132678us
I20260709 12:15:05.147161 28000 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:15:05.104534 27729 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.152844 27980 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:05.148177 27934 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:05.179134 27955 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:05.208549 27890 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.209725 28015 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:05.212095 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 780) took 1003 ms. Trace:
I20260709 12:15:05.217013 27782 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:05.203300 23463 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:05.223955 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:05.250170 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:05.260535 23468 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:05.260766 23316 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:05.248950 24014 rpcz_store.cc:276] 0709 12:15:04.208336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.208532 (+   196us) service_pool.cc:224] Handling call
0709 12:15:05.211893 (+1003361us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b5400083754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:05.212029 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589427968}
I20260709 12:15:05.222234 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:05.228750 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:05.237481 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.262768 28005 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:05.265635 27987 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.266279 28019 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.266183 27987 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.258472 28015 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:05.267704 28015 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.268957 28015 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:05.248260 27955 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:05.271036 27729 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.200832 27956 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:05.248296 27934 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.273725 27956 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:05.274989 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95730.3us
I20260709 12:15:05.275785 27934 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.258710 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:05.275745 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:05.177462 27961 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.255324 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:05.279433 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 127042us
I20260709 12:15:05.280045 23313 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:05.290388 23394 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:05.292698 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.293414 23316 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 6.
I20260709 12:15:05.296183 23468 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:05.248238 27985 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:05.291802 27549 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.668s	user 0.008s	sys 0.004s
I20260709 12:15:05.295589 28000 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:05.313772 27549 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.314702 28000 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.317302 28017 raft_consensus.cc:2764] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:15:05.318300 27953 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:05.296555 27961 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.324811 27961 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.327586 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:05.329088 27549 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:05.329295 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1923) took 1307 ms (client timeout 1519 ms). Trace:
I20260709 12:15:05.265069 27813 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.259807 27782 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:05.329854 23396 rpcz_store.cc:269] 0709 12:15:04.021453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.021559 (+   106us) service_pool.cc:224] Handling call
0709 12:15:05.329087 (+1307528us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b5400196ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:05.329216 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1935712512}
I20260709 12:15:05.308523 23467 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:15:05.309561 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.300858 27934 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:05.306072 24014 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:15:05.307453 27987 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:05.306583 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51410us
I20260709 12:15:05.434587 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:05.330925 23395 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:04.355020 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:05.297824 28005 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:05.473493 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 173349us
W20260709 12:15:05.486657 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52186.2us
I20260709 12:15:05.487669 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:05.506541 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:05.498051 23462 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:05.475541 27971 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.607350 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:05.602921 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 60577.2us
I20260709 12:15:05.501772 27899 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.512244 27976 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:15:05.564216 27909 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.724s	user 0.005s	sys 0.007s
I20260709 12:15:05.576146 27729 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.502317 27997 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:05.621990 28009 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:05.627588 27890 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:05.629839 28025 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:05.630327 27919 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:05.638561 27979 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.641891 23461 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:05.642650 27586 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:05.696353 27586 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.698764 27586 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.645328 27897 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:05.699555 27897 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.646513 27959 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:05.647368 27901 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.002s
I20260709 12:15:05.501960 28005 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.430354 27793 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:05.650892 28011 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 12:15:05.651919 28022 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.575975 27794 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:05.655616 27955 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.656145 27911 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:05.497804 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:05.657784 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:05.744891 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 87197.6us
I20260709 12:15:05.620049 28023 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:15:05.666301 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:05.545192 27884 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.340829 27985 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.515522 27987 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.673537 27729 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.668381 27782 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.543823 27549 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.125s
I20260709 12:15:05.691174 27813 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:05.692798 27914 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.736s	user 0.005s	sys 0.007s
I20260709 12:15:05.725082 23392 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:05.714510 27901 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.689100 27890 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.702220 27919 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.740744 28005 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.701861 27897 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.742949 27793 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.616186 27971 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.746524 23316 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:05.746883 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:05.751024 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 253285us
W20260709 12:15:05.721721 28007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.000s	sys 0.014s
I20260709 12:15:05.755668 28029 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:05.743517 28022 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.749759 27794 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.753042 27900 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.753199 27962 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:05.790419 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.790927 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:05.819604 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.798028 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 291502us
W20260709 12:15:05.812819 28007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.000s	sys 0.015s
I20260709 12:15:05.809787 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:05.684293 27997 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.752764 27911 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.762889 27987 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.785579 27729 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.821724 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:05.799278 27782 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:05.766038 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:05.831887 27793 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:05.835487 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:05.857614 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.822762 27962 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:05.860661 23314 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:05.846275 27897 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.887009 27794 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.887324 27987 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.887526 23465 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:05.889580 27997 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.889876 27968 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.862426 27549 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:15:05.891040 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2588) took 1022 ms. Trace:
I20260709 12:15:05.891407 23462 rpcz_store.cc:276] 0709 12:15:04.868215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.918977 (+ 50762us) service_pool.cc:224] Handling call
0709 12:15:05.890824 (+971847us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:05.890930 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329373440}
I20260709 12:15:05.892305 27890 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:05.885370 27637 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.004s
I20260709 12:15:05.817776 28029 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.854800 28007 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:15:05.863937 27993 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.864066 23470 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:15:05.866320 27817 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.870564 27919 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.872169 28022 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.835367 28033 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:05.861920 27962 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:05.873777 27911 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.875898 27985 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.878189 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2276) took 1250 ms. Trace:
I20260709 12:15:05.854156 28002 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:15:05.926858 23463 rpcz_store.cc:276] 0709 12:15:04.627385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.628014 (+   629us) service_pool.cc:224] Handling call
0709 12:15:04.628438 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:05.259988 (+631550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.877920 (+617932us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b540044af54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:05.878125 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1268390144}
I20260709 12:15:05.878446 27729 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:05.853473 27914 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:05.856819 27813 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 12:15:05.853901 28015 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.006s
I20260709 12:15:05.824323 27900 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.893420 27955 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:05.929853 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 321139us
I20260709 12:15:05.941059 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:05.922308 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:15:05.893525 27971 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.893392 27637 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:05.861088 27793 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.957548 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1022) took 1081 ms. Trace:
I20260709 12:15:05.958411 27933 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:05.958668 27813 consensus_queue.cc:1243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:05.959097 27985 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.959698 27985 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.959932 27817 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:05.960600 27817 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:05.962570 27985 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.961796 27817 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.963167 28029 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.963466 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:05.951644 27987 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.964694 27987 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.965123 27900 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:05.943059 27777 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.958876 27933 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:05.966912 27987 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.965940 27777 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.967876 27817 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:05.968043 27793 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:05.969077 27777 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:05.969947 24007 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:05.896585 28005 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:15:05.967116 27933 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:05.971868 27933 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:05.982296 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:05.982985 24014 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:05.926337 27995 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:05.985975 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45045.8us
I20260709 12:15:05.985728 27995 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.987934 27995 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:05.985741 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:05.958166 24010 rpcz_store.cc:276] 0709 12:15:04.865412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.865683 (+   271us) service_pool.cc:224] Handling call
0709 12:15:04.874346 (+  8663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.874564 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:05.239061 (+364497us) log.cc:844] Serialized 14 byte log entry
0709 12:15:05.946235 (+707174us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:05.946313 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:05.946382 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.946478 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:05.946654 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6796,"prepare.queue_time_us":384262,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":223562,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":21986,"thread_start_us":972909,"threads_started":3,"wal-append.queue_time_us":569038}
I20260709 12:15:05.995867 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:05.951794 28039 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.997654 28039 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.000770 27777 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:05.938400 28033 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:06.002653 24014 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:06.003602 28033 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.941288 27729 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:05.918787 27993 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:06.006549 28026 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:15:06.015338 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:05.985226 27953 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.666s	user 0.008s	sys 0.004s
I20260709 12:15:05.896417 27897 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:06.019181 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.021636 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:05.953400 28038 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:15:06.028995 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 194918us
I20260709 12:15:06.043498 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.045197 23388 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:06.047883 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:06.051532 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 192303us
I20260709 12:15:06.051831 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.012202 27993 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.001288 28039 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:06.064658 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.068033 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.070796 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:06.071729 23389 raft_consensus.cc:2376] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14. Re-sending same reply.
I20260709 12:15:06.026829 27909 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:05.952183 27890 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:15:06.076274 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.082659 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 786) took 1045 ms. Trace:
I20260709 12:15:06.027441 27841 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.027168 27729 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.068941 23470 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:06.081135 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.081601 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:06.078111 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.088158 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.089562 23465 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:06.089737 23316 raft_consensus.cc:2368] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
I20260709 12:15:06.089730 23463 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:06.083667 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.084846 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:06.080811 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.077239 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.091086 23394 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:06.028649 27777 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:06.085790 24016 rpcz_store.cc:276] 0709 12:15:05.036980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.045685 (+  8705us) service_pool.cc:224] Handling call
0709 12:15:05.074339 (+ 28654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:05.074568 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:05.308709 (+234141us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.081518 (+772809us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:06.081577 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:06.081623 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.082320 (+   697us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:06.082598 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19923,"prepare.queue_time_us":192039,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":143755,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4288,"thread_start_us":999391,"threads_started":3,"wal-append.queue_time_us":545511}
I20260709 12:15:06.050786 27647 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:06.095420 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2558) took 1104 ms. Trace:
W20260709 12:15:06.097961 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.102022 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.107107 27897 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.074458 28046 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.109493 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:06.110579 23393 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:06.109561 23389 raft_consensus.cc:1217] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.10->[13.11-16.13]   Dedup: 13.11->[14.12-16.13]
I20260709 12:15:06.114447 23389 pending_rounds.cc:85] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 11
I20260709 12:15:06.114859 23389 pending_rounds.cc:107] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:15:06.115198 23389 raft_consensus.cc:2889] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:06.126199 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:06.126986 23470 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.116991 27647 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.110096 23387 rpcz_store.cc:276] 0709 12:15:04.991404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.991580 (+   176us) service_pool.cc:224] Handling call
0709 12:15:04.993549 (+  1969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:04.994021 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:04.994054 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:05.704932 (+710878us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:05.705449 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:15:05.896853 (+191404us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:05.896891 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:05.896958 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:06.095054 (+198096us) raft_consensus.cc:1685] finished
0709 12:15:06.095078 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:06.095344 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":499,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":35739,"spinlock_wait_cycles":3522304,"thread_start_us":182863,"threads_started":1,"wal-append.queue_time_us":337882}
I20260709 12:15:06.128695 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:06.061842 28017 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:15:06.129398 24009 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.084674 27909 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.130283 24010 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:06.129841 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.131244 24008 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.131493 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.132143 27992 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.132227 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.132649 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.109716 27777 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:06.132848 28054 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:06.134276 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.134910 24012 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:06.116520 27637 consensus_queue.cc:1059] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:15:06.092284 27933 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.136444 27637 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:15:06.136865 27637 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:06.117784 27681 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:15:06.133240 27777 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:06.135677 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:06.137559 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 455769us
I20260709 12:15:06.138479 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:06.132860 24008 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:06.142282 27782 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:15:06.142442 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.143414 23316 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:06.143426 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.086987 28040 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:06.144485 23319 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:06.087888 28007 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:06.160111 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:06.143644 28055 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.135483 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1823) took 1050 ms. Trace:
I20260709 12:15:06.137635 27681 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:06.149935 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.142731 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.151791 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.168357 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:06.167253 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.152254 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:06.152918 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:06.126235 27934 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
I20260709 12:15:06.163923 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:06.164135 23390 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:06.163988 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:06.171962 23387 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:06.165777 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.173501 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.174238 24008 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:06.175982 24016 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:06.164801 23312 rpcz_store.cc:276] 0709 12:15:05.085308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:05.087314 (+  2006us) service_pool.cc:224] Handling call
0709 12:15:06.135301 (+1047987us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54003afa54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:06.135413 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176129792}
I20260709 12:15:06.177198 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.177853 23470 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.179136 23436 leader_election.cc:400] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:06.154953 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:06.179476 23436 leader_election.cc:403] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:06.171064 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.170090 24009 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.180941 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.182262 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.181820 24010 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 8: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:06.182896 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:06.137758 27637 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.182644 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:06.183817 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:06.247515 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.411734 28056 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:06.412379 28056 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:06.413465 28056 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.207089 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.414871 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2578) took 1559 ms. Trace:
I20260709 12:15:06.415594 23469 rpcz_store.cc:276] 0709 12:15:04.855945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.856182 (+   237us) service_pool.cc:224] Handling call
0709 12:15:04.856550 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:04.910670 (+ 54120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.414466 (+1503796us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b54002a08d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:06.414781 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114702080}
I20260709 12:15:06.237558 23391 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:06.418174 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:06.414907 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 232069us
I20260709 12:15:06.419445 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:06.420071 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.419567 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.420917 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:06.322214 24011 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:06.241539 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.186141 27817 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:06.427275 28041 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.239460 24013 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:06.310765 23387 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:06.421307 24008 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:06.426257 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 243794us
I20260709 12:15:06.420459 27744 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:06.429625 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:06.412524 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:06.348925 27959 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
W20260709 12:15:06.327728 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 148720us
I20260709 12:15:06.211445 23394 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:06.235184 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2555) took 1249 ms. Trace:
I20260709 12:15:06.464354 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:06.466068 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:06.470276 23469 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.164445 27992 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:06.472399 23391 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:06.471201 23395 rpcz_store.cc:276] 0709 12:15:04.986170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.987018 (+   848us) service_pool.cc:224] Handling call
0709 12:15:05.330722 (+343704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:05.435548 (+104826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.234869 (+799321us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400196ed0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:06.235091 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":789892352}
I20260709 12:15:06.456419 24013 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:06.164842 28029 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:06.246686 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.477303 24013 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.478827 24013 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.479534 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.480180 23466 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:06.482091 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.482832 23466 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.417217 27955 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:06.170787 27681 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:06.631947 27955 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.632370 27955 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.632606 27681 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.633682 27681 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.217329 27900 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:06.355743 28022 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.637697 27900 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.638160 27900 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.637837 28022 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.484344 24015 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 7.
I20260709 12:15:06.407668 28042 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.181514 28051 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:06.447635 28023 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.224021 28009 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:06.397998 27921 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.166531 27897 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.300s
I20260709 12:15:06.347380 27993 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:06.401641 27971 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.337479 28000 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.180938 27934 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:06.664274 27934 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:06.664801 27934 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.666885 27934 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.166612 28054 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.310s
I20260709 12:15:06.668322 23461 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:06.668035 28000 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:06.669091 28057 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.322s	user 0.000s	sys 0.003s
I20260709 12:15:06.171525 27782 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.319s
I20260709 12:15:06.468919 27744 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.496085 24013 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:06.474735 28053 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:06.354224 27962 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.497449 28064 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.483950 27637 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:15:06.171890 28055 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.488353 27992 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.168715 28007 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.184218 27953 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.311s
I20260709 12:15:06.468892 27817 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.489478 28045 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:15:06.519169 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:06.550419 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 78657.2us
I20260709 12:15:06.671242 27971 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.495383 28039 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 12:15:06.567070 28003 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.467s	user 0.011s	sys 0.015s
I20260709 12:15:06.474478 28041 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.570078 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.571664 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:06.573101 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.714308 23462 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:06.568562 27777 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.590047 27961 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:06.600090 27987 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:06.604205 23392 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2549) took 2401 ms (client timeout 1835 ms). Trace:
I20260709 12:15:06.577165 27647 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.607245 27909 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.608343 28019 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:06.617074 27816 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:06.627816 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 444012us
I20260709 12:15:06.631122 23469 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.656759 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.640187 28022 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.561790 28073 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:06.670346 28000 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.642735 28051 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.670591 23461 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:06.166718 27933 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.318s
I20260709 12:15:06.641485 28049 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.660790 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 201548us
W20260709 12:15:06.687757 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:06.692332 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 200653us
I20260709 12:15:06.670845 27993 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.685230 27744 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.693856 28023 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.656843 28042 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:06.690059 27958 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:15:06.683787 27934 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.700735 23390 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 12:15:06.686964 27841 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:06.696484 28053 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:06.659330 27921 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.706383 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:06.549273 28060 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.147s	user 0.000s	sys 0.002s
W20260709 12:15:06.708453 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:06.761339 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45300.4us
W20260709 12:15:06.763563 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57226.6us
I20260709 12:15:06.764267 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:06.695456 27901 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.719247 28039 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:06.720769 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.703677 28046 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.726493 23392 rpcz_store.cc:269] 0709 12:15:04.202354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:04.341327 (+138973us) service_pool.cc:224] Handling call
0709 12:15:06.603862 (+2262535us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54002ea254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:06.604048 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3026387712}
W20260709 12:15:06.761093 28070 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:15:06.729468 27961 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.738004 23466 raft_consensus.cc:2410] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:15:06.735950 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:06.694768 28064 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.747876 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:06.729696 27959 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.733666 28019 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:06.781957 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46098.7us
I20260709 12:15:06.783215 27971 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.783805 27971 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:06.784174 27971 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.784664 27971 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:06.758342 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:06.712373 24014 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:06.786680 28042 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.765465 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:06.787346 28053 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.788611 27993 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:06.755967 28023 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.790012 28041 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.767560 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.791551 28023 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:06.732427 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:06.705520 28045 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.792631 28023 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:06.792834 28045 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:06.797634 27921 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.747257 28073 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.798600 28073 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:06.799643 27777 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.783238 28019 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.800985 27992 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:06.770561 28017 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:06.734481 27987 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.740878 27816 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.775358 28046 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.768617 28039 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:06.750195 28051 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:06.803642 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55832.5us
I20260709 12:15:06.806951 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.768991 28075 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:06.831969 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 73678.1us
I20260709 12:15:06.842693 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:06.843562 23465 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:06.843896 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:06.844650 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.765394 27934 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.837108 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:06.840767 24011 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:06.839390 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:06.837335 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71974.1us
I20260709 12:15:06.829710 28023 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:06.751051 28049 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.838265 28000 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:06.840564 28074 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.854837 27816 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.855223 27744 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.902918 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.903247 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.899852 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:06.902918 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.904002 23321 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:06.846400 28042 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:06.904327 23315 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:06.858193 28039 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.860487 28046 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.842568 28075 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:06.911932 23389 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:06.912613 23396 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:06.848508 27890 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.838352 27909 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.898108 27782 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:06.871488 28061 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.482s	user 0.002s	sys 0.000s
I20260709 12:15:06.871728 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.877725 27971 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.879390 27980 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.947132 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:06.881894 28054 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:15:06.830974 27901 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:06.888573 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.973131 27744 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:06.892670 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.829075 28017 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:06.973551 28037 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.129s	user 0.003s	sys 0.000s
I20260709 12:15:06.973865 28017 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:06.895076 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:06.895684 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56364.4us
I20260709 12:15:06.969125 27961 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:06.847034 28070 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:15:06.899893 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.900859 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:06.839064 28081 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:06.904815 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.906818 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.906998 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.991967 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:06.902328 23466 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:06.829640 27959 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.909219 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:06.894848 28023 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:06.863885 28045 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.919283 24012 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:06.920271 23465 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:06.909757 27816 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:06.900386 28025 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:06.921415 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:06.879623 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.019759 23389 raft_consensus.cc:2368] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
W20260709 12:15:07.020228 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 140788us
I20260709 12:15:06.951505 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.923111 27890 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.944543 27909 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:06.959489 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:06.960562 28084 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:06.962723 27987 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:06.967025 28079 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:06.967922 28039 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:06.921558 27817 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:06.970407 27971 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.947261 27980 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.978303 27782 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:06.980221 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.044657 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:06.980554 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:06.914743 28075 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.982535 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:06.839671 28055 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.027490 28025 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.899501 27900 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:06.901825 28000 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:06.986155 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 91190.4us
I20260709 12:15:06.988776 23391 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:06.913938 27962 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:06.998363 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:06.990357 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:06.999907 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.064085 23470 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:07.005163 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:07.064515 23467 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:06.987886 27986 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.010921 27934 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:07.015826 28078 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:07.071933 28086 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:15:07.073206 27986 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.073891 27909 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.075477 27986 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.075740 27900 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.026381 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:06.991861 28002 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:07.027820 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.078769 27782 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.030493 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.080498 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:07.081101 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.081653 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.082131 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 813) took 1014 ms. Trace:
I20260709 12:15:07.083142 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.082566 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.083705 23319 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.083856 23461 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:06.889302 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:07.084795 23267 auto_rebalancer.cc:893] Removing replace marker failed: Replica da1e2fd157894634bf92d6c82c06ec64 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:07.085150 23313 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:07.085305 23267 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:15:07.041929 23387 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:07.036410 28084 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.087814 27934 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:07.088433 28084 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.089963 27647 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:07.045897 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.047418 23464 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.091485 28019 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.092135 23321 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:07.094074 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.096643 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.097584 23465 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:07.082549 23388 rpcz_store.cc:276] 0709 12:15:06.067669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.067936 (+   267us) service_pool.cc:224] Handling call
0709 12:15:06.069340 (+  1404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:06.070491 (+  1151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:06.070538 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:06.071205 (+   667us) log.cc:844] Serialized 26 byte log entry
0709 12:15:06.287576 (+216371us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:06.326626 (+ 39050us) log.cc:844] Serialized 14 byte log entry
0709 12:15:06.326904 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:06.326981 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:06.587522 (+260541us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b4800002dd8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:15:06.599794 (+ 12272us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b1800333ca0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:06.870774 (+270980us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:07.081919 (+211145us) raft_consensus.cc:1685] finished
0709 12:15:07.081940 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:07.082075 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35349,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1000,"raft.queue_time_us":275,"raft.run_cpu_time_us":3232,"raft.run_wall_time_us":516806,"spinlock_wait_cycles":60251904,"thread_start_us":170014,"threads_started":1,"wal-append.queue_time_us":255454}
I20260709 12:15:07.027967 28081 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.050890 27959 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.051005 27890 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.053222 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.113721 23395 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:07.059077 28073 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.005195 27961 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:15:07.039258 28040 consensus_meta.cc:287] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.868s	user 0.013s	sys 0.000s
I20260709 12:15:07.065618 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:07.077927 28000 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.079157 23389 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:07.056742 27816 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:07.085696 23267 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:15:07.038009 28079 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.131426 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.048058 23396 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:06.905272 28074 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:07.089069 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 62787us
I20260709 12:15:07.132467 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.100544 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.136248 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.106130 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.139200 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.140117 23312 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:07.109925 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.142988 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:07.143251 23313 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:07.110755 28081 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.112113 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.161060 23469 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:07.112522 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.113603 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.022071 28007 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.114982 23318 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:07.119628 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.120275 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.121315 28040 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:15:07.057767 28051 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.149914 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.168882 23470 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:15:07.150394 27995 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:15:07.171439 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.172065 23395 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:07.141160 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.155148 28073 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.155908 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:07.156631 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:07.158878 28002 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.161732 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.162384 23461 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:07.162788 23389 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:07.163808 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98155us
I20260709 12:15:07.164582 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.165000 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.121855 27980 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.167250 23319 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:07.185130 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.185916 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.187108 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.169165 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:07.188055 23396 raft_consensus.cc:606] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.188367 23396 raft_consensus.cc:614] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:06.923954 27955 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.173843 23312 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:07.188956 23436 leader_election.cc:400] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
I20260709 12:15:07.189253 23436 leader_election.cc:403] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:15:07.164307 27729 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.163085 28081 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.178395 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:07.167640 27681 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.182905 27817 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:15:07.183776 24011 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:15:07.201498 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:07.202029 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.199838 23312 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:07.204500 23313 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:07.193614 28041 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.132205 28064 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.204449 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:07.174839 23388 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:07.194010 27959 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:07.206451 23393 raft_consensus.cc:2368] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:07.194602 28042 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.216794 23395 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:15:07.217509 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.190246 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:07.217875 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:07.221606 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.210223 28055 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:07.200969 28084 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:07.210999 28041 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.220151 27959 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.200049 27729 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.220422 27681 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:07.210394 28000 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:15:07.226933 27993 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:07.242408 28045 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:07.245678 28041 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.252076 28029 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:15:07.252712 28007 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:07.263075 28007 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.264161 28054 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:07.264559 28054 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.226198 28082 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:07.272202 24012 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:07.222420 27816 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:07.252086 24015 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:07.235349 28084 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.274003 28082 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.265493 28054 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.274899 27816 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:07.275020 28007 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.269341 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.237604 27980 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:07.300407 27980 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.300534 28054 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.301487 27980 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.310196 27729 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:07.310916 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.317605 28075 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.297216 27816 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:07.320000 28075 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.321015 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.296685 28084 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.322018 28074 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.250185 28002 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.322968 28079 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.323688 27817 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:07.277101 28051 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:07.278162 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.223278 28064 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 12:15:07.282806 28088 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:15:07.248134 27959 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:07.340260 28042 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
I20260709 12:15:07.321821 28023 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.227093 27647 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:15:07.342442 27647 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:07.342522 27992 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:07.342850 27647 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.343055 28023 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.321826 28007 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.343835 27647 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.345013 27647 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.348832 23467 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:07.350028 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.298938 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.248229 28055 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.327124 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.352185 23469 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.329463 23391 raft_consensus.cc:2410] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:07.258044 27681 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.259349 27955 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.357769 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:07.358511 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59705.1us
I20260709 12:15:07.330998 28082 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.295862 28000 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:07.359354 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 138465us
W20260709 12:15:07.362296 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.363974 23469 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:07.365015 23434 leader_election.cc:400] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
I20260709 12:15:07.367522 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.372285 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.376598 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.376770 24008 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:15:07.380610 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.376297 28051 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:15:07.382642 23465 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:07.382817 28051 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:07.383371 28051 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.383908 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.385592 27729 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:07.386360 27729 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.386587 23470 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:15:07.387894 23470 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:15:07.387537 27729 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.389068 23470 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:15:07.390018 23466 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:15:07.395784 27959 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.397276 24014 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:07.401659 28039 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:07.406083 28084 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.406987 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.407315 24012 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:07.408341 28084 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.410097 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.385195 23465 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.414084 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.343011 27992 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:07.415736 27992 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.416971 27992 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.417825 27681 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.423017 23466 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:15:07.423604 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.414671 23467 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.433945 23469 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:07.441649 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:07.432785 23465 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:07.446420 27959 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.448042 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.448937 23465 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:07.442474 24010 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:07.449853 28042 raft_consensus.cc:476] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Not starting pre-election -- already a leader
W20260709 12:15:07.453837 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 463557us
I20260709 12:15:07.454319 27987 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:07.455025 27987 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:07.455120 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.456053 27987 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.457470 27986 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:15:07.454047 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.443805 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:07.460016 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.438251 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.457747 27909 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:07.469756 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.469883 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:07.470603 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 370133us
I20260709 12:15:07.471449 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.472785 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.474974 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:07.475543 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.475800 23466 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:07.373653 28077 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:07.384668 28051 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.474625 27961 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.484710 28079 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.485611 27955 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:07.475075 27909 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:07.485158 27841 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:07.487685 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.488544 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.530155 23392 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:07.530295 28092 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.376s	user 0.000s	sys 0.003s
I20260709 12:15:07.489238 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.525542 27961 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.531317 23388 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:07.489902 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.532124 23396 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:07.490541 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.532739 27961 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.492302 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:07.524482 28079 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.533638 28079 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.534518 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.535657 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.493637 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:07.537880 23391 raft_consensus.cc:606] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.538764 23391 raft_consensus.cc:614] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:07.495157 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.495882 27971 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:07.496712 28073 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:07.540901 28049 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:07.510990 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:07.524184 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 234220us
I20260709 12:15:07.374985 28000 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.528294 27841 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.543849 27841 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.544277 28000 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.421072 28055 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.479437 28074 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:15:07.550622 28017 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 12:15:07.551185 28017 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.552822 24009 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:07.553963 23468 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:07.469347 28041 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.555982 28079 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:07.556556 28093 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.398s	user 0.003s	sys 0.000s
I20260709 12:15:07.557175 24016 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:07.557327 28041 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.559060 28079 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:07.559578 28094 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.401s	user 0.000s	sys 0.004s
I20260709 12:15:07.560864 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.563294 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.564260 23461 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.567087 28017 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.528062 27909 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:07.572871 27959 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.574877 28081 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.578061 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2566) took 1407 ms. Trace:
I20260709 12:15:07.597760 23465 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:07.548990 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:07.599978 23391 raft_consensus.cc:606] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:07.583737 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.558s (SENT) [suppressed 14 similar messages]
I20260709 12:15:07.600345 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:07.584731 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89677.4us
I20260709 12:15:07.600365 23391 raft_consensus.cc:614] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:07.587016 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.601668 23314 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:15:07.592495 23467 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:07.594265 27997 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.586359 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:07.598778 23393 rpcz_store.cc:276] 0709 12:15:06.171128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.186039 (+ 14911us) service_pool.cc:224] Handling call
0709 12:15:07.577846 (+1391807us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b54002e9854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:07.577999 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1791969536}
I20260709 12:15:07.606807 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.608242 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.585253 28051 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:07.610474 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.612259 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.586457 27890 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:07.590106 27900 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:07.587121 27934 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.598201 27959 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.595695 28073 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.632539 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.610165 27980 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.628288 23464 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:07.625118 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:07.596118 28074 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:07.618304 27890 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.593757 27961 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:07.620273 27919 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:07.618158 28049 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:07.633111 28073 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.629163 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.616323 27997 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.627792 28071 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.627351 27793 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:07.634150 27959 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.624645 27637 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:15:07.610628 27962 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:15:07.620464 28064 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:15:07.624220 28029 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:15:07.673337 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.645364 28019 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:07.653945 27890 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:07.655063 27955 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:07.605202 23461 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:07.637635 28005 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:15:07.688638 24007 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:07.673522 28073 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.674726 28007 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.683890 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.634534 27934 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.678853 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.639076 27647 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:15:07.676944 28079 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:07.707826 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 75404.1us
I20260709 12:15:07.674690 28049 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.721737 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:07.742816 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 64058.7us
I20260709 12:15:07.637519 28002 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:15:07.690043 28029 consensus_queue.cc:1243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:15:07.681963 28019 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.744287 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:07.679913 27919 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:07.646449 27980 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.676857 28071 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.740782 27900 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:07.752496 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.753304 23316 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:15:07.690436 27793 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:07.745954 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:07.710647 27955 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:07.753821 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.757196 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:07.757747 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.757943 28077 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:07.754029 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.755198 23314 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:07.739866 28079 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.764717 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:07.765950 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.766319 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.766793 23321 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:07.761847 28074 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.767231 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.767481 28042 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:07.757583 27955 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.766021 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.770008 23319 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:07.747381 27953 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:07.745569 27971 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:07.772137 23287 leader_election.cc:400] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:07.780576 23287 leader_election.cc:403] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:07.772321 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.749245 28019 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.784278 27637 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:07.784502 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:07.800729 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.755012 27900 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:07.761124 28077 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:07.811367 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.774678 28079 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:07.822788 27971 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:07.825867 28042 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:07.870712 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:07.871330 23320 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:15:07.829998 28028 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:15:07.830361 28101 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:15:07.832476 27817 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.834945 28025 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:07.836284 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1083) took 1031 ms. Trace:
I20260709 12:15:07.838117 28054 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:07.843297 28102 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:15:07.844723 23462 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:15:07.844161 27919 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.775913 27953 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:07.902639 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:07.843369 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:07.787580 28100 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
I20260709 12:15:07.772032 28097 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:07.860293 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.862289 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:07.867223 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.909648 23312 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.910689 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:07.872737 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:07.911800 28099 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 12:15:07.788374 28064 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:07.912974 28064 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:07.913254 28105 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:15:07.913386 28064 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:07.913836 28064 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:07.914480 28084 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:07.915025 28084 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.763631 28005 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:15:07.875958 28106 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:15:07.875263 23393 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:07.884088 28002 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:15:07.918531 28002 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:07.918918 28002 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.919761 28002 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.921008 28002 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.775851 28076 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:15:07.921403 28005 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:07.921871 28005 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.883147 24009 rpcz_store.cc:276] 0709 12:15:06.805197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:06.807471 (+  2274us) service_pool.cc:224] Handling call
0709 12:15:06.807760 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:07.553303 (+745543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.835977 (+282674us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b540074e854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:07.836208 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712954752}
I20260709 12:15:07.812286 28077 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:07.878072 27816 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.001s
I20260709 12:15:07.874800 28028 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.879424 28042 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.853935 28039 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:07.917680 23313 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:07.812714 27900 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:07.923182 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:07.871716 27993 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:07.925194 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.936767 23391 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:07.938180 27729 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:07.925733 23218 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:07.927860 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.928939 23317 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:07.898789 27903 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:15:07.909302 27953 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:07.946106 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:07.887293 27817 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.930941 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.950397 23389 raft_consensus.cc:2393] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:07.933413 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.887775 28054 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:07.887573 27971 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:07.909360 23314 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:07.930511 28025 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:07.917285 28084 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:07.952785 23391 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:07.930724 23312 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:07.855095 28049 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.976965 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:07.936072 27816 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:07.995889 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:07.998167 23320 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:15:07.942863 27993 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:07.942903 28039 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.000538 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:08.004400 28071 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.005002 27971 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.002399 28111 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:08.017936 28111 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:08.018366 28111 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:07.970932 27841 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.022941 27841 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.023928 23395 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:08.025802 23390 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:08.025940 28107 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:15:08.028262 28071 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:08.028748 28071 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:08.029148 28071 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.034660 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:08.038760 28042 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.039520 28054 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.040858 28054 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:08.041723 28109 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.125s	user 0.003s	sys 0.000s
I20260709 12:15:08.003997 28025 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:08.044490 23218 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 10, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.046314 23218 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:07.951648 28081 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:07.954783 27903 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:08.000193 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 255982us
I20260709 12:15:08.049300 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:08.047714 23218 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:08.050199 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 265757us
W20260709 12:15:08.051306 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 113590us
I20260709 12:15:08.067507 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.068413 23469 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:07.941323 28005 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.077906 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:08.078738 23320 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:08.089228 23469 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:07.997889 28076 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:15:08.091127 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.094955 28113 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:08.056540 27841 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:15:08.098968 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.057704 27903 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:08.112289 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.080644 27986 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:15:08.100102 28106 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:15:08.118227 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:08.119611 24013 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:08.126381 23464 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:08.114164 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.127982 23469 raft_consensus.cc:2376] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10. Re-sending same reply.
I20260709 12:15:08.125671 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:07.994411 28028 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.131713 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:08.135532 28116 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:08.135381 27903 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.113108 27955 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.114897 28005 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:08.146261 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.144948 23464 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:08.146661 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:08.165423 23469 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
W20260709 12:15:08.166018 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 76479.1us
I20260709 12:15:08.133947 28113 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:15:08.167018 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:08.166407 23320 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:08.107262 27647 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:15:08.100095 28077 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.182404 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:08.137620 27841 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:08.135653 27817 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.151149 28017 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.172161 28113 consensus_queue.cc:1243] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:08.131415 27793 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:08.190963 27903 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:08.144711 27986 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:08.147548 28116 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:08.197837 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.198594 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.263561 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:08.199224 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:08.182192 27934 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:08.184974 28028 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:08.293503 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:08.203197 23360 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:08.206669 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:08.295665 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 89061.1us
I20260709 12:15:08.203064 28111 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.207754 27993 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.325042 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.325982 23392 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:08.209584 23395 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:08.191574 28055 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:08.210495 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43535.9us
W20260709 12:15:08.213799 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:08.334115 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.218001 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.335350 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:08.221977 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.336225 23465 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:15:08.223151 27817 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:08.198933 28064 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.224216 27992 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.002s
I20260709 12:15:08.224982 28039 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.228477 28041 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.151113 28025 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.233188 27959 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.001s
W20260709 12:15:08.235770 23219 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 219) took 31862 ms (client timeout 15000 ms). Trace:
I20260709 12:15:08.236548 23218 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 7, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 7 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:08.358136 23218 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.237476 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:08.238077 28106 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:08.360232 28106 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:08.239818 28131 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:08.361927 28131 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:08.240599 28132 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:08.362910 28132 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.211865 28123 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:08.244618 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.365660 28123 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:08.366010 23470 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:08.249042 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:08.251041 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:08.252799 23393 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:15:08.361194 28106 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.161223 27955 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.368902 28106 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:08.256048 23316 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:08.369598 23316 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:08.370436 23316 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.371199 27955 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.254858 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:08.371609 23316 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.263965 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.373724 23468 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:08.259893 24015 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 12:15:08.263700 23464 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
W20260709 12:15:08.294203 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95042.8us
I20260709 12:15:08.297353 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.265027 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:08.250504 28094 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:15:08.295414 24007 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:08.324405 23315 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:08.377815 23315 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:08.292915 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:08.338618 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da1e2fd157894634bf92d6c82c06ec64 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.338698 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.380223 23466 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:08.380606 23467 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:08.339064 23392 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:08.339478 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:08.384044 23469 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:08.387432 28121 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:08.388566 28121 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:08.389616 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.365283 28132 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:08.342123 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.344527 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.391294 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:15:08.349332 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 146216us
I20260709 12:15:08.393857 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:08.393918 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.354295 23389 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:08.394659 23392 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:08.366169 28123 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:08.378799 23315 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.395242 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.395594 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.395947 23315 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:08.208781 27841 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.385018 23218 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.396673 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.404918 23218 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:08.356117 28041 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.418418 23395 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:08.335361 27934 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.335395 28083 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:08.410065 23218 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.241343 28019 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.326185 28076 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:08.414728 27959 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.417922 27986 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.328357 27816 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.422516 23313 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:08.257242 27953 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.335908 28113 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:08.465513 28119 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.468752 23316 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:15:08.469149 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74364.8us
I20260709 12:15:08.469666 23218 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.412990 28123 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.483978 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 186571us
I20260709 12:15:08.487288 23315 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:08.490442 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:08.424918 28073 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.000s
I20260709 12:15:08.356848 27971 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:15:08.527129 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.425601 28083 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:08.566380 24014 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 8: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:08.597535 28145 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:08.357564 23219 rpcz_store.cc:269] 0709 12:14:36.379760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.383241 (+  3481us) service_pool.cc:224] Handling call
0709 12:15:08.235682 (+31852441us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:03.647286 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:03.647873 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:03.647906 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:15:03.647925 (+    19us) tablet.cc:662] Decoding operations
0709 12:15:03.648567 (+   642us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:03.648651 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:03.648677 (+    26us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:15:03.649240 (+   563us) tablet.cc:701] Row locks acquired
0709 12:15:03.649258 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:15:03.649406 (+   148us) write_op.cc:270] Start()
0709 12:15:03.649549 (+   143us) write_op.cc:276] Timestamp: P: 1783599303649355 usec, L: 0
0709 12:15:03.649578 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 12:15:03.650245 (+   667us) log.cc:844] Serialized 2148 byte log entry
0709 12:15:03.898257 (+248012us) op_driver.cc:464] REPLICATION: finished
0709 12:15:04.073926 (+175669us) write_op.cc:301] APPLY: starting
0709 12:15:04.081317 (+  7391us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:04.198138 (+116821us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:04.210510 (+ 12372us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:04.244878 (+ 34368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:04.245093 (+   215us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 12:15:04.245139 (+    46us) write_op.cc:312] APPLY: finished
0709 12:15:04.245375 (+   236us) log.cc:844] Serialized 57 byte log entry
0709 12:15:04.260901 (+ 15526us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:04.272064 (+ 11163us) write_op.cc:454] Released schema lock
0709 12:15:04.323221 (+ 51157us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:04.360052 (+ 36831us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:04.361167 (+  1115us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":649450,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":158263,"apply.run_cpu_time_us":9661,"apply.run_wall_time_us":283051,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":173491,"prepare.run_cpu_time_us":3329,"prepare.run_wall_time_us":6117,"replication_time_us":227550,"thread_start_us":202107,"threads_started":2}]]}
I20260709 12:15:08.598433 28145 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.596771 28073 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.491309 28113 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:08.566846 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62681.4us
I20260709 12:15:08.571885 24013 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:15:08.493815 27900 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.004s
I20260709 12:15:08.356680 28084 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.138s
W20260709 12:15:08.539001 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.468943 28078 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:08.579156 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.595619 23396 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:08.597070 28123 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.597638 28083 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:08.641323 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:08.566066 23218 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 13, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 13 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.575892 28074 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:08.621935 24015 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:08.629709 27637 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:08.628747 27971 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:08.629015 28106 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.587471 27997 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.494611 27793 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.565407 28054 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:08.637643 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 147322us
W20260709 12:15:08.575310 27909 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.968s	user 0.007s	sys 0.015s
I20260709 12:15:08.587764 28019 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.645423 28145 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:08.646147 27782 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:08.646407 28113 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.631261 28041 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.647300 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:08.653357 28002 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.001s
I20260709 12:15:08.651855 28142 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:08.474345 27934 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.476480 28116 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.627745 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.531345 28071 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.622983 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:08.576151 27816 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.088s
W20260709 12:15:08.657991 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:08.789273 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:08.735973 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 402923us
I20260709 12:15:08.736174 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:08.661721 23218 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:08.775447 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:08.671769 23319 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:15:08.791574 23319 pending_rounds.cc:85] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:15:08.791936 23319 pending_rounds.cc:107] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:15:08.792196 23319 raft_consensus.cc:2889] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:08.791464 23218 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:08.781939 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:08.708921 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:08.595649 28127 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:15:08.803730 23391 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:08.805398 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 69319.2us
I20260709 12:15:08.807770 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.806566 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:08.810700 23319 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:15:08.811081 23319 pending_rounds.cc:85] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:08.811296 23319 pending_rounds.cc:107] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:15:08.811520 23319 raft_consensus.cc:2889] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:08.811926 27959 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.812599 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:08.813853 28071 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.814224 23321 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:15:08.814602 23388 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:08.813364 23315 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 8: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:08.815511 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.809286 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.816254 23391 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:08.816751 23463 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:08.817713 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:08.818151 23463 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 12:15:08.711938 28079 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.838s	user 0.005s	sys 0.005s
W20260709 12:15:08.710045 28029 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:15:08.815133 28084 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:15:08.809508 27841 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:15:08.825563 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:08.826354 27903 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:15:08.828300 24009 raft_consensus.cc:2410] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:15:08.829728 23466 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:15:08.748493 28106 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:08.756441 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 109213us
I20260709 12:15:08.843614 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:08.632998 27900 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.642282 28078 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.682011 28017 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:08.658252 28064 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:08.658975 28077 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:08.686817 27980 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:08.682880 28121 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.789590 27986 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:08.660389 28111 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.579183 28119 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.764418 28130 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.178s	user 0.003s	sys 0.000s
W20260709 12:15:08.764663 28125 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.523s	user 0.001s	sys 0.000s
I20260709 12:15:08.748373 27909 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.768751 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.761139 27955 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:08.748675 28054 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.770191 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:08.770833 28113 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:08.772418 28007 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.050s	user 0.007s	sys 0.002s
I20260709 12:15:08.751403 27782 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:08.748535 28074 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.788470 24008 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:08.815196 27901 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:15:08.793370 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:08.794169 23218 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.715662 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:08.753913 27637 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.818681 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:08.819527 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:08.883985 23218 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.830595 23287 leader_election.cc:400] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:08.830832 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:08.629839 28073 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:08.658927 27890 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.957s	user 0.005s	sys 0.005s
W20260709 12:15:08.690197 28129 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:15:08.778095 27971 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.774134 28083 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.804605 28028 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.776423 23467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:15:08.814451 28002 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:08.892204 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48668.8us
W20260709 12:15:08.854190 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.893409 23218 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:08.895004 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 64559.5us
I20260709 12:15:08.856247 23466 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:08.925880 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.829233 27817 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.829970 28142 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:08.957204 23391 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:08.961911 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:08.962167 23467 raft_consensus.cc:606] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:08.963078 23218 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 11, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:08.978222 23218 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:08.827678 28132 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:08.886101 27955 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:08.974179 23387 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:08.891384 28123 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:15:08.859041 27900 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.937739 27782 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:08.861291 28025 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:15:08.859151 28106 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:08.988612 24011 raft_consensus.cc:2368] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15.
W20260709 12:15:08.991132 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.987701 23393 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:08.893087 28084 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:08.940414 28007 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:08.829345 28079 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:08.954468 27959 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:08.954741 28111 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:15:08.994871 28125 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:15:08.954442 28121 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:08.941402 27890 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.089721 23318 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:08.986919 27817 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:08.999845 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:08.962316 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:09.163210 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 200900us
I20260709 12:15:08.999248 23218 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 13, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 13 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:09.166676 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 382981us
I20260709 12:15:09.166901 23218 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 7 to 10, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 10 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:08.993844 28151 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 12:15:08.987950 28064 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:08.991499 27955 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:09.054486 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72510.4us
I20260709 12:15:08.938659 27986 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:08.842250 27903 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.143s
I20260709 12:15:09.140373 23392 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:09.190629 23469 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:08.989398 27980 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:15:09.193787 24015 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:15:09.194388 24015 pending_rounds.cc:85] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 10
I20260709 12:15:09.194697 24015 pending_rounds.cc:107] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:15:09.194937 24015 raft_consensus.cc:2889] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:09.171921 27647 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:15:09.205947 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.208204 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.209770 23391 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:09.212280 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:08.981261 28073 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 12:15:08.993212 28145 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:15:08.975131 28142 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:09.172787 23218 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:09.229231 23218 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:09.178164 27901 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:09.068786 28111 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:09.150655 28150 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:15:09.275913 28157 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:15:09.280663 28147 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:15:09.060796 28084 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.289322 27890 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.183210 23467 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:09.184142 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:09.292908 24008 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:09.227041 28017 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.293761 28017 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.293736 28084 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.296334 28017 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.251345 28022 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.297801 28022 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.184211 27793 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:15:09.187641 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:08.992098 23470 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:08.983882 28028 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.141744 28119 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.222074 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:09.069193 27959 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.227221 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.227641 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.314082 23312 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:09.246449 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.247754 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.248770 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.216960 28106 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.250607 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.251708 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.190137 27934 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:09.301221 24008 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:15:09.160737 28130 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:15:09.191526 27900 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:15:09.310702 23314 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:09.246935 28142 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.253008 28111 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:09.316846 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 94874.9us
I20260709 12:15:09.323813 28028 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.325757 27637 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.326179 27959 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
W20260709 12:15:09.326627 28146 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.569s	user 0.012s	sys 0.000s
I20260709 12:15:09.326817 28106 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.331790 28071 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:09.333928 23319 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:09.336649 28083 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.337045 27817 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.337642 28078 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.338007 27986 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.425027 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1966) took 1362 ms. Trace:
I20260709 12:15:09.338529 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.428362 23321 raft_consensus.cc:2410] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:15:09.338786 28079 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.339247 28074 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.339563 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.442134 24012 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:15:09.343573 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1832) took 1592 ms. Trace:
I20260709 12:15:09.343787 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 805) took 1139 ms. Trace:
I20260709 12:15:09.443318 24013 rpcz_store.cc:276] 0709 12:15:08.204364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.204527 (+   163us) service_pool.cc:224] Handling call
0709 12:15:08.204849 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:08.625178 (+420329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.343537 (+718359us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400083754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:09.343729 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843533696}
I20260709 12:15:09.443167 23318 rpcz_store.cc:276] 0709 12:15:07.751336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.751452 (+   116us) service_pool.cc:224] Handling call
0709 12:15:07.751744 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.091207 (+1339463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.343321 (+252114us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002747d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:09.343505 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3076667136}
I20260709 12:15:09.333510 24010 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:15:09.348340 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.351807 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:09.355194 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.357605 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.446736 23320 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:09.326876 28118 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:09.334218 28150 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 12:15:09.375761 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:09.421856 23463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.035s	sys 0.074s
W20260709 12:15:09.448727 23463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.035s	sys 0.074s
W20260709 12:15:09.448882 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.227286 28132 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:15:09.346093 28145 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:09.406863 28071 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:09.437789 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.447435 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:09.425805 23390 rpcz_store.cc:276] 0709 12:15:08.063003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.083182 (+ 20179us) service_pool.cc:224] Handling call
0709 12:15:08.083534 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:08.087093 (+  3559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.087133 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:08.094396 (+  7263us) log.cc:844] Serialized 52 byte log entry
0709 12:15:08.197543 (+103147us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:08.244723 (+ 47180us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.309319 (+ 64596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.309454 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.763741 (+454287us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:09.418696 (+654955us) raft_consensus.cc:1685] finished
0709 12:15:09.424793 (+  6097us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.424968 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255122,"prepare.queue_time_us":56440,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":84720,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":36483,"spinlock_wait_cycles":8131328,"thread_start_us":100671,"threads_started":1,"wal-append.queue_time_us":476849}
I20260709 12:15:09.453244 24013 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:15:09.462316 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.463557 23321 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:09.355916 28022 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.460368 27934 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.468463 27934 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.470960 27934 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:09.466280 28131 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.471786 28131 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.473958 28131 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.474761 28097 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.475474 28097 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:09.476245 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.433229 27953 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.421475 27637 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.422396 28028 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.481704 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.478698 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.378393 28054 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.484747 28042 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:09.484563 27782 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.454066 28145 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.482635 24012 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:09.487860 23390 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:15:09.385057 27909 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.448046 28117 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:09.452015 28121 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.491608 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.504743 23389 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:09.504875 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.505738 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.492621 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1090) took 1244 ms. Trace:
I20260709 12:15:09.506980 24014 rpcz_store.cc:276] 0709 12:15:08.248618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.248747 (+   129us) service_pool.cc:224] Handling call
0709 12:15:09.492407 (+1243660us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54006301d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.492565 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455933824}
I20260709 12:15:09.492882 24015 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:09.493690 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2583) took 1243 ms. Trace:
W20260709 12:15:09.495728 23466 consensus_meta.cc:287] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.000s
I20260709 12:15:09.509021 23388 rpcz_store.cc:276] 0709 12:15:08.250635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.252133 (+  1498us) service_pool.cc:224] Handling call
0709 12:15:09.493471 (+1241338us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54000a8554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.493614 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1008904960}
I20260709 12:15:09.498708 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.509508 23466 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:15:09.511685 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2571) took 2051 ms. Trace:
I20260709 12:15:09.512226 23390 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:09.513037 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2622) took 1180 ms. Trace:
I20260709 12:15:09.512081 23394 rpcz_store.cc:276] 0709 12:15:07.460756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.465652 (+  4896us) service_pool.cc:224] Handling call
0709 12:15:07.638129 (+172477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:07.638487 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:07.638517 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:07.638946 (+   429us) log.cc:844] Serialized 26 byte log entry
0709 12:15:07.747616 (+108670us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:07.747653 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:07.747739 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:08.041011 (+293272us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:08.044035 (+  3024us) raft_consensus.cc:1685] finished
0709 12:15:08.044057 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.511527 (+1467470us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001973d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.511634 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":675,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3202,"spinlock_wait_cycles":362479360,"thread_start_us":74782,"threads_started":1,"wal-append.queue_time_us":247884,"wal-append.run_cpu_time_us":4748,"wal-append.run_wall_time_us":1145634}
I20260709 12:15:09.513406 23466 rpcz_store.cc:276] 0709 12:15:08.332251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.855731 (+523480us) service_pool.cc:224] Handling call
0709 12:15:08.856113 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.510319 (+654206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.512985 (+  2666us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:15:09.522946 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.424970 27647 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:09.490599 28042 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.447988 28118 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:09.497579 28145 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:09.551456 24014 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:09.561417 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.552556 28097 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:09.553104 28164 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:15:09.538331 27647 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.534137 27817 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.500633 28071 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.562119 24013 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:09.563659 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:09.491019 27637 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.490013 28077 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:15:09.568217 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.491169 28028 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.592496 24012 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
W20260709 12:15:09.594369 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 364357us
I20260709 12:15:09.571188 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.577555 28113 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.536695 28042 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:09.580835 28106 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:09.581552 28163 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:15:09.583439 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2617) took 1432 ms. Trace:
I20260709 12:15:09.583760 27637 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.587205 28097 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.588613 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2621) took 1256 ms. Trace:
I20260709 12:15:09.589198 28132 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:09.568503 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:09.620890 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52567.6us
I20260709 12:15:09.623721 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:09.596647 23390 raft_consensus.cc:2368] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:15:09.601190 23461 rpcz_store.cc:276] 0709 12:15:08.151116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.151286 (+   170us) service_pool.cc:224] Handling call
0709 12:15:08.151660 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:08.161715 (+ 10055us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:15:08.326949 (+165234us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.748814 (+421865us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:08.757933 (+  9119us) log.cc:844] Serialized 52 byte log entry
0709 12:15:08.758095 (+   162us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:08.779210 (+ 21115us) log.cc:844] Serialized 14 byte log entry
0709 12:15:08.779464 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:08.779554 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.583005 (+803451us) raft_consensus.cc:1685] finished
0709 12:15:09.583035 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.583340 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38295,"prepare.queue_time_us":460531,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":8560,"raft.queue_time_us":18697,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":37466,"spinlock_wait_cycles":31356800,"thread_start_us":447301,"threads_started":2,"wal-append.queue_time_us":619993}
I20260709 12:15:09.513711 28117 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.605547 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:09.514752 28121 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.593834 27681 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.631168 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:09.636047 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.636677 24007 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:09.637197 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:09.631521 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.613022 23463 rpcz_store.cc:276] 0709 12:15:08.331895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.821040 (+489145us) service_pool.cc:224] Handling call
0709 12:15:08.821320 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:08.833128 (+ 11808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:08.833168 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:08.856659 (+ 23491us) log.cc:844] Serialized 26 byte log entry
0709 12:15:09.449145 (+592486us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:09.449185 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.449251 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:09.588311 (+139060us) raft_consensus.cc:1685] finished
0709 12:15:09.588341 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:09.588527 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16169,"raft.queue_time_us":11469,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":19331,"thread_start_us":592085,"threads_started":1,"wal-append.queue_time_us":729486}
I20260709 12:15:09.640753 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.643051 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.643762 23320 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:09.643884 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.644469 23318 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.644797 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.647642 23461 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:15:09.648018 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1087) took 1419 ms (client timeout 1708 ms). Trace:
W20260709 12:15:09.648427 24009 rpcz_store.cc:269] 0709 12:15:08.228464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.237170 (+  8706us) service_pool.cc:224] Handling call
0709 12:15:09.647824 (+1410654us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54004e3254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.647959 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076250880}
I20260709 12:15:09.650100 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 825) took 1058 ms. Trace:
I20260709 12:15:09.650625 23392 rpcz_store.cc:276] 0709 12:15:08.593539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.593793 (+   254us) service_pool.cc:224] Handling call
0709 12:15:08.594124 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.189831 (+595707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.340557 (+150726us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5400261720. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a9677a02 00007ff0a9673dd6 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:09.649873 (+309316us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:09.650040 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1290447360}
I20260709 12:15:09.652436 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1957) took 1778 ms. Trace:
I20260709 12:15:09.652966 23387 rpcz_store.cc:276] 0709 12:15:07.874245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.874361 (+   116us) service_pool.cc:224] Handling call
0709 12:15:07.876452 (+  2091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:08.988640 (+1112188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.652142 (+663502us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540005a854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:09.652362 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2524006144}
W20260709 12:15:09.660315 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 1962) took 1728 ms (client timeout 1703 ms). Trace:
W20260709 12:15:09.660989 23396 rpcz_store.cc:269] 0709 12:15:07.932416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:07.933157 (+   741us) service_pool.cc:224] Handling call
0709 12:15:09.660025 (+1726868us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54003449d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.660215 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481315584}
I20260709 12:15:09.668888 24012 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:15:09.669633 24012 pending_rounds.cc:85] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 10
I20260709 12:15:09.669970 24012 pending_rounds.cc:107] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:15:09.670287 24012 raft_consensus.cc:2889] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:09.672055 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:09.673244 23466 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:09.677174 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.677043 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.681936 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:09.682695 24014 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:09.683776 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:09.684489 23461 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:15:09.684832 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.685218 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:09.620251 28132 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.678617 28119 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:09.742482 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61686.4us
I20260709 12:15:09.734763 28151 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:15:09.735070 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.743317 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:09.735839 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:09.749338 23468 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:09.738003 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.751371 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.738790 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:09.752476 24016 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:09.630617 28019 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.594369 27817 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.607980 28113 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.685669 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.707775 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:09.767637 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 283558us
I20260709 12:15:09.742408 23389 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:09.745424 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:09.777798 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:09.705052 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:09.749037 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.779009 23387 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:09.757972 23318 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:09.748010 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.780350 23388 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:09.677093 28135 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:09.763831 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.739880 27934 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:09.708410 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.629806 28041 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:09.767988 23392 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:09.655815 28007 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:09.680078 28077 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:09.812124 28077 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:09.813083 28077 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.744410 28132 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.815289 27953 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:09.815794 27953 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.816138 27953 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:09.743981 27900 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.817795 28168 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:09.818004 27900 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.821061 27900 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.738487 28064 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.783197 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:09.821739 28064 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.822499 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.823819 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:09.818006 27782 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:09.825004 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:09.825682 24011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1092) took 1576 ms (client timeout 1669 ms). Trace:
W20260709 12:15:09.826098 24011 rpcz_store.cc:269] 0709 12:15:08.249215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.250808 (+  1593us) service_pool.cc:224] Handling call
0709 12:15:09.825470 (+1574662us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b540044ff54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:09.825620 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426125696}
I20260709 12:15:09.826683 28071 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:09.827740 28029 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.828244 28029 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.635457 28106 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:15:09.785950 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.788700 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.627674 28097 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.793509 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.679693 27681 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.794379 23468 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:09.796804 23321 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:09.810099 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.685945 28073 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.821300 28116 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:09.825517 27782 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:09.784353 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:09.869236 23466 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:09.869663 23465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.006s	sys 0.043s
W20260709 12:15:09.870021 23465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.006s	sys 0.043s
I20260709 12:15:09.815322 28077 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:09.833781 23315 raft_consensus.cc:2410] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:15:09.834479 23318 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:09.676229 27637 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.147s
I20260709 12:15:09.842595 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:09.787783 28135 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.845165 28097 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:09.787622 27959 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:15:09.832949 28106 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:09.849792 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.850934 28113 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.852524 28132 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.756800 28119 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.686348 27955 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.831315 23320 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:09.945416 23320 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:15:09.945789 23320 pending_rounds.cc:85] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 5
I20260709 12:15:09.946022 23320 pending_rounds.cc:107] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:15:09.946229 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:09.947433 23315 raft_consensus.cc:2410] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:09.892889 23466 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:09.804829 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:09.829650 28078 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:09.847950 28111 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.849829 27817 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:09.940755 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 264918us
I20260709 12:15:09.919585 28151 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.809173 27934 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:09.966235 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:09.970036 23387 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:09.946265 23320 raft_consensus.cc:2889] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:09.932271 23389 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:09.972393 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:09.929589 23318 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:09.934474 28135 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:09.869166 28073 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:09.988605 23389 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:09.978456 23319 raft_consensus.cc:2368] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13.
W20260709 12:15:09.989900 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 211241us
I20260709 12:15:09.986070 23320 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:15:09.984939 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:09.991788 23392 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:15:09.818351 28168 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.967861 28175 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:09.972195 28113 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:09.993300 23320 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:15:09.986667 27681 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:09.980001 28078 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:09.629913 28118 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.939131 28106 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:10.013146 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 363337us
I20260709 12:15:09.988452 27961 raft_consensus.cc:2764] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:10.018193 23395 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 12:15:10.018718 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 189284us
W20260709 12:15:10.078133 23433 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.012s	sys 0.017s
I20260709 12:15:10.078509 23394 raft_consensus.cc:606] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:09.989470 28117 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.079631 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:10.080539 23389 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:10.008289 27817 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.078615 23433 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.012s	sys 0.017s
I20260709 12:15:09.987278 28151 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:09.991173 28028 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.073639 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.097318 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.099234 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.087083 23466 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:09.995584 28081 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:10.019703 28097 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:09.997942 28121 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.071471 28029 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.109889 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:09.949349 28142 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 12:15:10.004671 28113 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:10.002223 27647 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.114349 23389 raft_consensus.cc:2410] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:10.085944 28180 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:10.123427 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:10.104183 28041 raft_consensus.cc:2764] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 12:15:10.129878 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.112272 28028 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.103835 28081 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:10.115387 28055 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:15:10.137919 28165 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:15:10.173534 23466 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:10.016225 28145 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.014744 28175 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:10.176178 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46412.2us
I20260709 12:15:10.181249 23463 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:15:10.181524 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.182130 23466 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:10.112038 28170 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:10.111972 28084 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:15:10.135754 28182 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:10.117893 28142 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.135916 28123 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:10.208847 23321 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:10.209656 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 218727us
I20260709 12:15:10.142833 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:10.185262 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:10.211315 23464 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 8.
I20260709 12:15:10.211751 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:10.101626 27817 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.105021 28042 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.197839 23463 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:10.227252 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:10.228250 23464 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:10.205181 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.229010 23461 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:10.193266 23466 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:10.176576 28055 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.234081 28055 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:10.201088 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 216216us
I20260709 12:15:10.213361 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:10.247584 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:10.219791 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.221390 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.249543 23469 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.194394 27900 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.197798 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.253885 23468 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:10.253816 27900 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.256461 27900 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.119304 28097 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:10.257315 27986 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:10.257896 27986 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:10.258805 27986 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.260706 28166 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:10.262780 28166 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:10.263250 28166 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:10.264094 28166 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.269513 28167 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:10.277470 28167 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:10.276861 23468 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:10.194483 28170 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.141717 28078 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.287273 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:10.200786 28054 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.300603 23316 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.325349 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:10.174342 28028 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.197758 28186 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:10.200743 28182 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.132359 28113 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.338299 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.338907 23461 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.302141 23469 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:10.137765 28180 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.224133 27971 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:10.209981 28181 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:10.291754 28079 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:15:10.329622 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 109987us
I20260709 12:15:10.348654 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.224869 28171 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:10.310679 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.340291 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.351824 23464 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.299829 28170 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.303181 28123 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:10.300868 28111 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.341396 23468 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
W20260709 12:15:10.336764 28173 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.339s	user 0.001s	sys 0.000s
I20260709 12:15:10.354853 23316 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:10.327781 28179 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.355831 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.432317 23463 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:10.322544 27980 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:10.438490 23461 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:10.348270 28168 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:10.402120 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81980.2us
I20260709 12:15:10.338094 28182 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:10.424985 23464 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:10.358438 27971 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:10.426975 23468 pending_rounds.cc:85] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 5
W20260709 12:15:10.351356 28172 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 12:15:10.428277 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:10.346601 28073 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:10.436012 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 87439.4us
I20260709 12:15:10.333611 28054 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.369719 28079 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:10.444973 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:10.447705 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.455818 23463 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:10.337692 28119 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.467850 23468 pending_rounds.cc:107] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:15:10.493611 23468 raft_consensus.cc:2889] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:10.362031 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.496536 23469 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:10.322260 28081 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.442435 28203 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.509730 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 164295us
I20260709 12:15:10.511536 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:10.512260 23468 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:10.450223 28167 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.518965 23314 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 12:15:10.519567 23314 pending_rounds.cc:85] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:10.519860 23314 pending_rounds.cc:107] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:15:10.520172 23314 raft_consensus.cc:2889] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:10.524165 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:10.528079 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.343225 28186 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.515447 28119 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.507094 28019 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.000s
I20260709 12:15:10.364403 28171 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:10.547066 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 102173us
I20260709 12:15:10.562861 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:10.477254 27841 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:10.483629 28151 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.532605 23362 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:10.468941 27980 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.562069 23468 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:10.487144 28123 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.535796 28189 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:10.568621 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2590) took 1361 ms. Trace:
I20260709 12:15:10.624082 23391 rpcz_store.cc:276] 0709 12:15:09.207174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.228842 (+ 21668us) service_pool.cc:224] Handling call
0709 12:15:09.229160 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:09.229371 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:09.229397 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:09.437679 (+208282us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:09.438105 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 12:15:09.842111 (+404006us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:09.865695 (+ 23584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.865801 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:10.568291 (+702490us) raft_consensus.cc:1685] finished
0709 12:15:10.568314 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:10.568559 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21429504,"thread_start_us":403665,"threads_started":1,"wal-append.queue_time_us":529950}
I20260709 12:15:10.555614 28081 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.444083 28179 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.660522 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 981) took 1854 ms. Trace:
I20260709 12:15:10.565881 28019 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:10.572705 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61518.6us
I20260709 12:15:10.627827 28142 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.540841 28073 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:15:10.661031 23465 rpcz_store.cc:276] 0709 12:15:08.806356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.170558 (+364202us) service_pool.cc:224] Handling call
0709 12:15:09.191496 (+ 20938us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:09.192048 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:09.192093 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:09.238353 (+ 46260us) log.cc:844] Serialized 52 byte log entry
0709 12:15:09.870383 (+632030us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:09.870420 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:09.870486 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:10.660262 (+789776us) raft_consensus.cc:1685] finished
0709 12:15:10.660292 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:10.660455 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":20539,"thread_start_us":561878,"threads_started":1,"wal-append.queue_time_us":690186}
W20260709 12:15:10.534516 28176 consensus_queue.cc:1175] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "54640ab8abeb4c949426d66c7e5aed87" responder_term: 13 status { last_received { term: 12 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:10.546880 28167 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.551810 28055 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:10.683671 28177 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.392s	user 0.001s	sys 0.001s
I20260709 12:15:10.636242 28189 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.554762 27900 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 12:15:10.632692 28181 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:10.695303 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica da1e2fd157894634bf92d6c82c06ec64 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:10.750689 23206 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 55456.3us
I20260709 12:15:10.572068 23469 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:10.701406 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1239) took 1057 ms. Trace:
I20260709 12:15:10.560339 28119 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.541119 28166 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:15:10.632849 27993 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:10.523136 28203 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.797574 28209 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:10.798071 28071 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.971s	user 0.014s	sys 0.000s
I20260709 12:15:10.798204 28209 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.800401 28209 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:10.800968 28209 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:10.802516 23463 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:10.802852 28167 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.787736 28192 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:10.803872 28206 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:15:10.804222 28192 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.686239 28175 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.724157 28028 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.634271 27971 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.835132 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1237) took 1208 ms. Trace:
I20260709 12:15:10.835975 23319 rpcz_store.cc:276] 0709 12:15:09.626369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.708098 (+ 81729us) service_pool.cc:224] Handling call
0709 12:15:10.834870 (+1126772us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400295254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:10.835054 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437595392}
W20260709 12:15:10.782070 28198 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 12:15:10.702178 28196 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 12:15:10.752159 23320 rpcz_store.cc:276] 0709 12:15:09.643976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.073592 (+429616us) service_pool.cc:224] Handling call
0709 12:15:10.074210 (+   618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:10.174842 (+100632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:10.174874 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:10.175433 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:15:10.347537 (+172104us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:10.347585 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.347664 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:10.701186 (+353522us) raft_consensus.cc:1685] finished
0709 12:15:10.701209 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:10.701344 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100128,"raft.queue_time_us":101907,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":16273,"spinlock_wait_cycles":13568,"thread_start_us":171736,"threads_started":1,"wal-append.queue_time_us":523964}
I20260709 12:15:10.714079 28111 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:10.891584 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.900327 23396 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:10.626816 27841 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:10.759277 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55823.2us
I20260709 12:15:10.840126 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.902670 23394 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:10.841987 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.904170 24008 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:10.844020 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.847175 28194 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:10.847419 28179 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:10.849391 28204 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:15:10.804725 28192 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.661464 28079 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:10.836575 27993 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:10.851447 28191 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
W20260709 12:15:10.902479 28077 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.994s	user 0.013s	sys 0.000s
W20260709 12:15:10.931164 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:10.733392 28189 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:10.930207 27993 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.863399 27817 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.865504 28054 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.930306 28142 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.942324 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.944888 28171 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.945777 28181 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.941869 28142 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:10.950466 28055 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.899899 28028 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.866613 28186 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:10.871757 28211 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:15:10.807348 28195 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:15:10.898195 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.907284 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:10.905998 23390 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:15:10.722806 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 160002us
I20260709 12:15:10.909031 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:10.911753 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.976962 24008 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:10.912784 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.921372 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.926666 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.978866 24011 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:10.930212 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:10.979729 24009 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
W20260709 12:15:10.801214 28135 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.000s
I20260709 12:15:10.977973 24015 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:10.859057 28180 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.939456 28187 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.902220 28111 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:10.884435 28209 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:10.903021 28042 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:10.980432 23466 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
W20260709 12:15:10.865286 28106 consensus_meta.cc:287] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.000s
W20260709 12:15:10.982200 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.016s	sys 0.048s
I20260709 12:15:10.983964 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:10.984215 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:11.032770 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.016s	sys 0.048s
I20260709 12:15:10.984583 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:10.983659 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:10.988575 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.037062 23391 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:10.999801 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.005342 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1106) took 1302 ms. Trace:
W20260709 12:15:11.007350 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.012s	sys 0.004s
W20260709 12:15:11.012364 23387 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.003s	sys 0.014s
W20260709 12:15:11.038877 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.012s	sys 0.004s
W20260709 12:15:11.038913 23387 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.003s	sys 0.015s
I20260709 12:15:11.038623 24010 rpcz_store.cc:276] 0709 12:15:09.702845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.734580 (+ 31735us) service_pool.cc:224] Handling call
0709 12:15:09.735026 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:09.936307 (+201281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:09.936342 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:09.974067 (+ 37725us) log.cc:844] Serialized 52 byte log entry
0709 12:15:10.133463 (+159396us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:10.133508 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.133585 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.005062 (+871477us) raft_consensus.cc:1685] finished
0709 12:15:11.005092 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.005272 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56687,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":24342,"thread_start_us":355789,"threads_started":2,"wal-append.queue_time_us":387584}
I20260709 12:15:11.033641 23396 raft_consensus.cc:2368] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:10.992030 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:10.993404 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:11.043828 23320 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:11.000499 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2486) took 1514 ms. Trace:
I20260709 12:15:11.044574 23318 rpcz_store.cc:276] 0709 12:15:09.485910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.986075 (+500165us) service_pool.cc:224] Handling call
0709 12:15:09.986376 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:09.986806 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:15:09.986834 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:09.987415 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:15:10.198304 (+210889us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:10.198364 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:10.198453 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.000289 (+801836us) raft_consensus.cc:1685] finished
0709 12:15:11.000312 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.000439 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88557,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3837,"thread_start_us":210298,"threads_started":1,"wal-append.queue_time_us":358678}
I20260709 12:15:11.046962 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:11.039435 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 559371us
W20260709 12:15:11.005990 28097 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.003s
W20260709 12:15:11.064257 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 531808us
W20260709 12:15:11.065940 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74005.8us
I20260709 12:15:10.996426 23980 leader_election.cc:400] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13.
I20260709 12:15:11.005810 28111 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:11.004261 28116 consensus_meta.cc:287] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.135s	user 0.010s	sys 0.001s
I20260709 12:15:11.000582 27993 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.066761 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:10.992542 28142 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.072451 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:11.141192 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 74487.7us
I20260709 12:15:11.066812 23980 leader_election.cc:403] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:11.068436 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.068277 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:11.145530 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 156290us
W20260709 12:15:11.145920 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 77412.4us
W20260709 12:15:11.139693 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.100224 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:11.070525 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.148497 23320 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:11.150171 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:11.151248 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:11.048688 28078 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:11.100612 28205 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:11.049703 27980 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.071499 28055 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.150854 23319 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:11.104611 28202 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:11.155282 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 84862.1us
W20260709 12:15:11.157356 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 88977.3us
I20260709 12:15:11.072536 28097 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.071916 28180 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.154640 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:11.165911 23982 leader_election.cc:400] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10.
I20260709 12:15:11.162567 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:11.163281 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.104717 28186 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:10.990687 28028 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.103956 28119 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:10.995087 28135 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.170291 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:11.172240 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:11.174976 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:11.152412 27681 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:11.153625 28181 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.177696 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.148613 27971 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:11.152669 27901 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:11.179059 23320 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:11.018646 28209 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.071357 28166 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.096s
I20260709 12:15:11.158984 28179 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:15:11.033223 28187 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.223587 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48711.7us
I20260709 12:15:11.158536 27993 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:15:11.224056 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 79492.4us
I20260709 12:15:11.064495 28106 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.172294 27953 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.356s	user 0.015s	sys 0.000s
I20260709 12:15:11.164373 28142 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:11.264194 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86576.5us
I20260709 12:15:11.266628 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.296645 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:11.299186 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.179246 28049 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:11.229637 28200 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:11.308573 28179 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:11.312813 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.316241 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.318035 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.319643 23318 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:11.161391 28078 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.325146 23312 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:11.326647 23470 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:15:11.327926 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.328109 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.329236 23470 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:11.329104 23464 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:15:11.173034 28202 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.272253 27901 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.297720 28097 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:11.311623 28049 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.341702 28135 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.342950 23465 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:11.344278 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.344966 23469 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 6.
I20260709 12:15:11.222584 27681 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.335404 28207 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.347141 28207 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.347607 28207 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.224886 28218 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:11.253545 28022 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.337325 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.339378 27901 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:11.358532 23464 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:11.179147 28186 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.349727 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.360932 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:11.361737 23468 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:11.351320 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.353812 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.172760 28055 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.323626 27993 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.302979 28208 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.393743 28208 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.394083 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:11.395103 28219 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:15:11.395270 23468 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:11.396570 28208 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.396503 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.323909 28180 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:11.415043 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.415779 23463 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:11.392037 28055 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.349363 27986 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:11.416793 27986 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:11.417798 27986 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.419698 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1973) took 2076 ms. Trace:
I20260709 12:15:11.360459 28132 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.341760 28049 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:11.393736 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.398242 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:11.422132 23464 raft_consensus.cc:2368] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:11.332718 28042 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:11.423013 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:11.423408 23469 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:15:11.423723 23464 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:11.402387 28224 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:15:11.404600 23462 raft_consensus.cc:2393] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:11.383199 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:11.406605 23316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2477) took 2735 ms (client timeout 1604 ms). Trace:
W20260709 12:15:11.432392 23316 rpcz_store.cc:269] 0709 12:15:08.670950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.715181 (+ 44231us) service_pool.cc:224] Handling call
0709 12:15:11.406384 (+2691203us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54003f98d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:11.406505 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3253345664}
I20260709 12:15:11.316144 27841 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.303793 27953 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.440142 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2596) took 1704 ms. Trace:
I20260709 12:15:11.313081 28142 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.406656 28170 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.987s	user 0.005s	sys 0.007s
I20260709 12:15:11.407922 28186 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.384773 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:11.415882 23468 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:11.457535 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.458417 24011 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:11.390281 23467 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:15:11.420153 23395 rpcz_store.cc:276] 0709 12:15:09.342943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.344865 (+  1922us) service_pool.cc:224] Handling call
0709 12:15:09.378881 (+ 34016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:10.078322 (+699441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.419412 (+1341090us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54000a8f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.419635 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469503360}
I20260709 12:15:11.425117 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.461966 23470 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:11.413183 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1226) took 2612 ms. Trace:
I20260709 12:15:11.440133 23465 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:11.406960 23318 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:11.416914 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.454247 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.457755 23468 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:11.450106 24015 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:11.465812 23463 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:11.359833 28218 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:11.462980 23313 rpcz_store.cc:276] 0709 12:15:08.800877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:08.800993 (+   116us) service_pool.cc:224] Handling call
0709 12:15:08.801267 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:11.149166 (+2347899us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:11.149207 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:11.149262 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:11.149300 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.411625 (+262325us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b18003a06c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419
0709 12:15:11.412893 (+  1268us) spinlock_profiling.cc:243] Waited 2.34 s on lock 0x7b5400148a54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.412945 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.413093 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3309,"raft.queue_time_us":3502,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":6211,"spinlock_wait_cycles":5402635264}
I20260709 12:15:11.440744 23387 rpcz_store.cc:276] 0709 12:15:09.735926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.985626 (+249700us) service_pool.cc:224] Handling call
0709 12:15:09.988124 (+  2498us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:09.991421 (+  3297us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:10.180442 (+189021us) log.cc:844] Serialized 14 byte log entry
0709 12:15:11.045484 (+865042us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:11.068192 (+ 22708us) log.cc:844] Serialized 52 byte log entry
0709 12:15:11.068349 (+   157us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:11.167540 (+ 99191us) log.cc:844] Serialized 14 byte log entry
0709 12:15:11.439785 (+272245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.439867 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.439909 (+    42us) raft_consensus.cc:1685] finished
0709 12:15:11.439921 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.440078 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74500,"prepare.queue_time_us":565050,"prepare.run_cpu_time_us":192,"prepare.run_wall_time_us":6327,"raft.queue_time_us":184138,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":28527,"spinlock_wait_cycles":573312,"thread_start_us":1235155,"threads_started":4,"wal-append.queue_time_us":957122}
I20260709 12:15:11.469583 23396 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:11.470875 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 204331us
I20260709 12:15:11.471076 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:11.475646 23388 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:15:11.476020 24010 raft_consensus.cc:606] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:11.476686 24010 raft_consensus.cc:614] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:11.478293 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.475466 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.481451 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.481578 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.482100 24016 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:11.482414 24009 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
W20260709 12:15:11.482996 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:11.483271 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 98620.6us
I20260709 12:15:11.485838 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.486526 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.490659 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.494174 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.495119 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:11.496548 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.454912 27890 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:11.454007 27971 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:11.453681 28179 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:11.505676 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.473255 28207 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.468127 28234 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:11.508530 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.507200 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.507513 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:11.522578 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.477499 28218 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:11.527874 23396 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:11.528195 23395 raft_consensus.cc:2410] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:15:11.530509 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.535697 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.544140 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:11.495662 27909 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:11.493758 28042 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:11.550490 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:11.544821 24016 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:11.551192 24007 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:11.502414 27890 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:11.503159 28186 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:11.498508 28071 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:11.555962 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:11.527256 28234 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.468998 28170 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.561086 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:11.543962 28222 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:15:11.564386 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:11.573055 23395 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:11.573326 28231 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:11.573839 28231 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.576668 27890 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.578238 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:15:11.554989 28182 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.071s	user 0.013s	sys 0.001s
I20260709 12:15:11.581463 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:11.550724 27909 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:11.596433 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 1977) took 2246 ms. Trace:
I20260709 12:15:11.597885 23393 rpcz_store.cc:276] 0709 12:15:09.350115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.514477 (+164362us) service_pool.cc:224] Handling call
0709 12:15:09.514793 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.635772 (+120979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:09.635806 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:09.635845 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:09.635879 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.596147 (+1960268us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003453d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.596194 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.596354 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277753600}
I20260709 12:15:11.527086 28179 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.603900 27890 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:11.612672 28241 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:15:11.604101 28232 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 12:15:11.607844 28225 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
I20260709 12:15:11.619112 23396 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:11.595211 28097 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:11.621701 28106 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.622002 28078 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:11.622713 28226 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
I20260709 12:15:11.561671 28186 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.625737 28187 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.626775 28186 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.572727 28234 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.629861 27971 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.575959 28231 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.581178 28028 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.645674 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.647812 24007 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:11.604836 27909 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:11.657877 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.659130 23467 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:11.571549 28071 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.664577 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 347449us
I20260709 12:15:11.673859 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:11.677505 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:11.605010 28182 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.699445 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 96029.8us
I20260709 12:15:11.701507 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:11.702131 23469 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:15:11.708668 28235 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:15:11.710206 24008 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:15:11.689222 28238 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:11.696681 28233 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:11.688925 28231 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:15:11.706183 28202 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.706887 28142 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.726305 28071 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.727672 28179 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.731225 28182 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.719713 28233 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:11.721066 28238 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:11.740515 28192 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.002s
I20260709 12:15:11.740942 28192 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.742270 28189 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.802s	user 0.010s	sys 0.002s
I20260709 12:15:11.743288 28192 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.738775 28234 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.745848 28234 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.742906 28189 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:11.746506 28181 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.001s
I20260709 12:15:11.746315 28238 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.746978 28181 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13.
I20260709 12:15:11.747910 28189 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.753747 28233 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.754525 28218 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.756742 28218 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:11.790408 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2499) took 1077 ms. Trace:
I20260709 12:15:11.791090 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2501) took 1078 ms. Trace:
I20260709 12:15:11.791451 23320 rpcz_store.cc:276] 0709 12:15:10.712311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.264643 (+552332us) service_pool.cc:224] Handling call
0709 12:15:11.265048 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:11.265603 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:11.265648 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:11.266262 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.629932 (+363670us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b1800449800. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:11.631947 (+  2015us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b1800449800. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:11.789104 (+157157us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:15:11.789241 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.789392 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.789530 (+   138us) raft_consensus.cc:1685] finished
0709 12:15:11.789582 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.790017 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":29126,"spinlock_wait_cycles":83438336,"thread_start_us":158371,"threads_started":1,"wal-append.queue_time_us":465588}
I20260709 12:15:11.794359 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2500) took 1081 ms. Trace:
W20260709 12:15:11.794725 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.794891 23316 rpcz_store.cc:276] 0709 12:15:10.712708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.433557 (+720849us) service_pool.cc:224] Handling call
0709 12:15:11.434012 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:11.434509 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:11.434550 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:11.435140 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.669407 (+234267us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:11.669482 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.669571 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.793650 (+124079us) raft_consensus.cc:1685] finished
0709 12:15:11.793694 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.794125 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9250,"thread_start_us":178340,"threads_started":1,"wal-append.queue_time_us":209313}
I20260709 12:15:11.791538 23313 rpcz_store.cc:276] 0709 12:15:10.713034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.471613 (+758579us) service_pool.cc:224] Handling call
0709 12:15:11.471933 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:11.472285 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:11.472318 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:11.472798 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.670828 (+198030us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:11.670877 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.670971 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.790540 (+119569us) raft_consensus.cc:1685] finished
0709 12:15:11.790592 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.790915 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12436,"thread_start_us":143591,"threads_started":1,"wal-append.queue_time_us":222897}
I20260709 12:15:11.806432 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:11.808383 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:11.812510 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:11.815073 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.816582 23470 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:11.818156 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:11.820825 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:11.825273 23395 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.823195 23461 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:11.831079 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.832280 23393 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:11.839533 27971 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.840615 27971 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.842516 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:11.844340 27971 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.816695 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.841107 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:11.845167 27971 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:11.852686 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.988997 23393 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:11.735805 28170 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:11.852933 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:11.854197 28207 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:11.856508 24007 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:11.735738 28049 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.989372 27971 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:11.991668 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.993040 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.872859 27955 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:11.873883 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:11.875715 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1243) took 1679 ms. Trace:
I20260709 12:15:11.878764 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2502) took 1153 ms. Trace:
I20260709 12:15:11.880074 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2481) took 2442 ms. Trace:
I20260709 12:15:11.883399 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.885056 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2595) took 2149 ms. Trace:
I20260709 12:15:11.886417 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2594) took 2198 ms. Trace:
I20260709 12:15:11.888911 27909 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:11.892933 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:11.894645 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:11.888935 23466 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2332) took 1582 ms (client timeout 1675 ms). Trace:
I20260709 12:15:11.898813 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2495) took 1166 ms. Trace:
I20260709 12:15:11.908892 27681 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.018764 23388 rpcz_store.cc:276] 0709 12:15:09.735621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.861660 (+126039us) service_pool.cc:224] Handling call
0709 12:15:09.903666 (+ 42006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:11.477985 (+1574319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.884453 (+406468us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54002ea254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.884851 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3614470912}
I20260709 12:15:11.911079 27934 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.913353 27953 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:12.021975 23466 rpcz_store.cc:269] 0709 12:15:10.306143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.310295 (+  4152us) service_pool.cc:224] Handling call
0709 12:15:11.888504 (+1578209us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b54003edad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:11.888704 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376241536}
I20260709 12:15:11.914812 28064 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:11.920006 23469 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 7.
I20260709 12:15:11.923738 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:11.925542 28029 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:11.926359 28142 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:11.926537 28189 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.946928 28118 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:11.959849 28042 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:11.962134 28131 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:11.832762 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:11.931285 28113 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:11.982237 23395 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:11.993886 28049 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:11.737632 28111 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.003s
I20260709 12:15:11.993716 27971 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.018975 23392 rpcz_store.cc:276] 0709 12:15:09.687720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.837512 (+149792us) service_pool.cc:224] Handling call
0709 12:15:09.837925 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:09.992379 (+154454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.885842 (+1893463us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54002dfd54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.886230 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353313024}
I20260709 12:15:11.914680 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1100) took 2297 ms. Trace:
I20260709 12:15:12.017590 23318 rpcz_store.cc:276] 0709 12:15:10.728087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.046603 (+318516us) service_pool.cc:224] Handling call
0709 12:15:11.878306 (+831703us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400148f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:11.878521 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625122688}
I20260709 12:15:12.018244 23321 rpcz_store.cc:276] 0709 12:15:09.437512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.842312 (+404800us) service_pool.cc:224] Handling call
0709 12:15:11.879542 (+2037230us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54002760d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:11.879832 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688695680}
I20260709 12:15:12.022127 23319 rpcz_store.cc:276] 0709 12:15:10.710906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.226959 (+516053us) service_pool.cc:224] Handling call
0709 12:15:11.303210 (+ 76251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:11.405913 (+102703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:11.405945 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:11.405988 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:11.406018 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.876861 (+470843us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54003c0d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.876922 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.877214 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":387,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":87,"spinlock_wait_cycles":235400576}
I20260709 12:15:12.045166 23393 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:12.017345 23317 rpcz_store.cc:276] 0709 12:15:10.195831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.222332 (+ 26501us) service_pool.cc:224] Handling call
0709 12:15:10.222616 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:10.349519 (+126903us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:10.349570 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:10.354321 (+  4751us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.409159 (+1054838us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b1800449860. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:11.468898 (+ 59739us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:11.469083 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.874990 (+405907us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400389d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:11.875070 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:11.875148 (+    78us) raft_consensus.cc:1685] finished
0709 12:15:11.875166 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:11.875441 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4935,"raft.queue_time_us":2292,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":72907,"spinlock_wait_cycles":302848768,"thread_start_us":58580,"threads_started":1,"wal-append.queue_time_us":1170867}
I20260709 12:15:12.042555 24012 rpcz_store.cc:276] 0709 12:15:09.617243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.617528 (+   285us) service_pool.cc:224] Handling call
0709 12:15:09.668372 (+ 50844us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:09.670882 (+  2510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:11.901228 (+2230346us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b54004e3250. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:11.914497 (+ 13269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78337792}
I20260709 12:15:12.050547 28049 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:12.050983 28049 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:12.051177 27901 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.013s
I20260709 12:15:12.051355 28049 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.051687 27901 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.061762 28042 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.064965 28042 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.048699 23391 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:12.076085 27955 raft_consensus.cc:2764] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:15:12.076893 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 244483us
I20260709 12:15:12.078926 27901 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.079707 27901 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:12.080415 28203 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:12.048681 28189 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.088892 28189 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.041527 28111 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:12.044129 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 120822us
I20260709 12:15:12.078823 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:12.049093 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 154893us
I20260709 12:15:12.099115 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1244) took 1901 ms. Trace:
I20260709 12:15:12.095557 28064 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:12.099640 23314 rpcz_store.cc:276] 0709 12:15:10.196097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.297675 (+101578us) service_pool.cc:224] Handling call
0709 12:15:10.297996 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:10.520728 (+222732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:12.092309 (+1571581us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540014a354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:12.097083 (+  4774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507039616}
I20260709 12:15:12.099480 28131 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:12.103344 23312 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1235) took 2480 ms (client timeout 2332 ms). Trace:
W20260709 12:15:12.106948 23312 rpcz_store.cc:269] 0709 12:15:09.609040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:09.734821 (+125781us) service_pool.cc:224] Handling call
0709 12:15:12.089004 (+2354183us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400149e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:12.089247 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3624524160}
I20260709 12:15:12.102835 28064 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:12.115197 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.124354 28209 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:12.125118 28131 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.125345 23387 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:12.092578 28029 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.118110 28064 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:12.131925 28123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.995s	user 0.014s	sys 0.026s
W20260709 12:15:12.132685 28123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.014s	sys 0.026s
W20260709 12:15:12.135674 28252 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.412s	user 0.002s	sys 0.000s
I20260709 12:15:12.119911 28189 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.107318 28142 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.126816 27955 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:12.151526 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:12.154364 23393 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:12.155400 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 76777.5us
I20260709 12:15:12.150096 27909 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:12.184224 27890 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.571s	user 0.022s	sys 0.000s
I20260709 12:15:12.232499 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:12.243904 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 95996.2us
I20260709 12:15:12.225297 28029 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:12.237934 28265 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.093s	user 0.000s	sys 0.002s
I20260709 12:15:12.252833 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:12.255718 28111 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.236361 28189 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:12.265538 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.268469 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:12.257126 28142 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.271257 27909 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:12.273056 23392 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:12.273231 27909 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:12.273741 27909 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:12.274458 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:12.276463 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.277160 24007 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:12.278175 24012 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:12.292717 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 67927.4us
I20260709 12:15:12.269945 27903 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.294916 27903 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.246353 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:15:12.297681 28246 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.574s	user 0.003s	sys 0.000s
I20260709 12:15:12.307762 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.320021 27900 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.320860 27900 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.322328 28145 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:12.315527 28263 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.104s	user 0.009s	sys 0.019s
I20260709 12:15:12.337210 28145 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:12.338224 28142 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:12.338400 23389 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:12.338783 28142 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:12.342490 28142 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:12.344306 28142 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:12.358192 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2605) took 1437 ms. Trace:
I20260709 12:15:12.355309 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:12.363549 27903 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.359980 23389 rpcz_store.cc:276] 0709 12:15:10.920141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.920694 (+   553us) service_pool.cc:224] Handling call
0709 12:15:12.356490 (+1435796us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54000a7b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:12.356679 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3099247360}
I20260709 12:15:12.368175 28189 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:12.348311 28049 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.391184 28049 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.377045 28064 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:12.409853 28166 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:12.410630 28166 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:12.410907 28264 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.256s	user 0.000s	sys 0.002s
I20260709 12:15:12.411041 28166 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:12.415051 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1248) took 1429 ms. Trace:
I20260709 12:15:12.415302 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:12.415504 23320 rpcz_store.cc:276] 0709 12:15:10.986107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:11.799222 (+813115us) service_pool.cc:224] Handling call
0709 12:15:11.799670 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:11.800104 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:12.023514 (+223410us) log.cc:844] Serialized 14 byte log entry
0709 12:15:12.414466 (+390952us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b1800284ce0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:15:12.414707 (+   241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:12.414756 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:12.414798 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:12.414874 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:12.414992 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":273346,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":936,"raft.queue_time_us":920,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":6289152,"thread_start_us":292498,"threads_started":2,"wal-append.queue_time_us":118803}
I20260709 12:15:12.417758 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1236) took 2796 ms. Trace:
I20260709 12:15:12.423012 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.423089 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:12.419174 23315 rpcz_store.cc:276] 0709 12:15:09.620855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:10.009268 (+388413us) service_pool.cc:224] Handling call
0709 12:15:10.077760 (+ 68492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:10.121152 (+ 43392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:10.121208 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:10.121876 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 12:15:11.939349 (+1817473us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b18004496e0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:12.111958 (+172609us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:12.116915 (+  4957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:12.346716 (+229801us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b5400388e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:12.346775 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:12.409979 (+ 63204us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:12.417330 (+  7351us) raft_consensus.cc:1685] finished
0709 12:15:12.417350 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:12.417543 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11946,"raft.queue_time_us":2753,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":4713,"spinlock_wait_cycles":162748288,"thread_start_us":90766,"threads_started":1,"wal-append.queue_time_us":2009889}
W20260709 12:15:12.431728 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 101847us
I20260709 12:15:12.432397 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:12.430187 28266 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:15:12.437714 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:12.439293 28207 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:12.439667 28207 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.441764 28207 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:12.460481 28145 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:12.470412 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:12.458549 28189 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:12.458884 27900 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.476348 28145 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.496321 27903 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:15:12.501214 28142 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:12.501876 28142 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.502307 28142 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:12.509892 23396 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:12.513278 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:12.488545 28064 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.496758 28145 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:12.488137 28189 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:12.519198 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65875.6us
I20260709 12:15:12.522104 27955 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:12.520457 27900 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.526554 27900 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.529860 28121 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.530716 28121 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.536190 28189 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.488832 27909 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.559213 28123 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:15:12.574400 28123 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:12.575687 28123 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:12.579577 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 90776.1us
I20260709 12:15:12.549901 27900 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.581961 23390 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:12.592454 28083 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:15:12.594537 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 94957.6us
I20260709 12:15:12.580929 28121 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.614446 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:12.627369 28271 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:15:12.639950 27909 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.643376 28064 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:12.643939 28064 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.644322 28064 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:12.652007 28166 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.663851 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:12.655546 28113 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.680161 23287 leader_election.cc:400] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:12.680557 23287 leader_election.cc:403] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:12.655858 28121 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:15:12.691984 23214 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:12.695578 23214 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 9 to 10, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 10 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:12.697809 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:12.679498 28113 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.700510 28113 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.701344 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.702231 23321 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:12.697554 28117 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:15:12.703697 28117 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:12.704102 28117 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:12.705010 28117 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.698556 23461 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.708547 28117 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:12.724753 28207 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.663596 27909 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:15:12.735286 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.729619 23214 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 12, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 12 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:12.731312 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74163.9us
I20260709 12:15:12.731442 23461 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:12.743111 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:12.750514 24008 raft_consensus.cc:2368] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:15:12.749842 23214 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 10, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 10 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:12.755061 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.755838 23320 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.741648 28113 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:12.762219 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:12.721243 28142 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.769474 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.770406 24012 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:12.763352 28166 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.792514 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.791704 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:12.796571 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:12.797405 23463 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:12.808418 23396 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:12.811506 23464 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:12.807462 23319 pending_rounds.cc:85] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:12.815608 28113 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:12.816030 28207 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.816629 28113 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:12.820187 28113 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:12.821277 28113 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:12.822690 28113 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:12.813546 23319 pending_rounds.cc:107] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 12:15:12.825858 23319 raft_consensus.cc:2889] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:12.799270 28189 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:12.826756 28189 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:12.828132 23315 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:12.827934 28189 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.831501 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:12.837774 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.838625 23313 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:12.839905 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.841416 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.855253 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:12.857133 23362 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.195:33909 timed out after 1.675s (SENT) [suppressed 18 similar messages]
I20260709 12:15:12.859328 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:12.862970 23390 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:12.869244 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:12.802743 28145 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:12.873100 23320 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:12.865545 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.880040 23463 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:12.882098 23320 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:12.882862 23313 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:12.882963 23467 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:15:12.882109 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.885689 28151 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:15:12.893847 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:12.880184 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.899273 23317 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:12.903540 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.904373 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:12.907258 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:12.909395 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.907280 23214 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:12.916421 27909 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:12.916854 27909 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:12.917213 27909 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:12.921103 23462 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:12.923712 28145 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.925041 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.874433 28064 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:12.915647 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43725.7us
I20260709 12:15:12.932320 23315 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:12.935757 23467 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:12.938616 28166 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:12.945672 28083 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:12.946395 28083 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:12.948877 28083 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:12.950737 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.951332 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.952090 23317 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:12.952337 24012 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:12.959973 28064 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:12.960673 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:12.961261 28042 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:12.963661 23320 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:12.937556 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:12.953212 23317 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:12.966566 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:12.974372 23390 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:12.979542 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:12.962091 28042 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:12.993781 23315 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:12.992794 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.001355 23313 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:12.977815 28064 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:13.000382 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.012619 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:12.986099 28207 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:13.026480 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.002315 28142 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:13.037531 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.041268 23362 leader_election.cc:400] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:15:13.009492 28064 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:13.038522 24008 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:13.043654 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:13.044521 23312 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:13.047914 23320 raft_consensus.cc:2376] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14. Re-sending same reply.
I20260709 12:15:13.061960 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.070940 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:13.073151 28237 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:13.071657 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.074067 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.051263 28064 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.077379 23314 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:13.076061 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.096248 24012 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:13.097245 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 85376.7us
I20260709 12:15:13.046051 28207 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.119813 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.089790 28286 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.130373 23470 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:15:13.130340 23214 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 8, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.128479 23462 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.138257 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.138938 23467 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:13.127218 28189 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:13.135425 28145 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:13.154660 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:13.150956 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50422.8us
I20260709 12:15:13.159150 24008 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:13.159173 23312 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:15:13.160336 23312 pending_rounds.cc:85] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:15:13.164151 23312 pending_rounds.cc:107] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:15:13.164438 23312 raft_consensus.cc:2889] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:13.174072 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.173779 28286 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.175772 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:13.176385 28286 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:13.177191 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.177551 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.178229 24015 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.178537 28049 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:13.122826 28042 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.181766 28299 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:15:13.183727 28299 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.117823 28237 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:13.192870 28290 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:15:13.194782 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.195513 23466 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:13.196812 23466 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:13.197826 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.198508 24014 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:13.182451 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.184620 28299 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.204093 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.181807 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.182085 28302 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:13.203544 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.210707 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.211380 23389 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:15:13.207664 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.208479 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.199836 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.209837 23390 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:13.223793 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46528us
I20260709 12:15:13.248728 23470 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.251530 28286 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:13.252002 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.251601 23465 pending_rounds.cc:85] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 7
I20260709 12:15:13.254334 23465 pending_rounds.cc:107] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:15:13.254606 23465 raft_consensus.cc:2889] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:13.255344 28299 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:13.256135 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.227403 23463 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:13.158902 28145 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:13.225729 23214 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 6 to 12, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:13.206528 23313 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:13.233506 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.238318 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.268656 23396 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:13.252049 28286 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:13.271734 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.207259 28189 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.290241 24009 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:13.291174 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.293679 23469 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:13.256574 28298 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.302448 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.248662 28302 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:13.305280 23393 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:13.305913 23390 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:13.309934 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.314997 23467 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:13.319586 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.318473 24015 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:13.320426 23462 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:13.322022 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.324149 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.330044 23394 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:13.331029 24014 raft_consensus.cc:2410] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:15:13.330406 24010 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:13.333886 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.334529 23395 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.313771 28303 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:13.343351 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 85319.4us
I20260709 12:15:13.344460 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.331753 28117 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:13.365741 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.342501 28207 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:13.438848 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.440549 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:13.446231 28287 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:15:13.368732 27900 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:13.369452 28117 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:13.437991 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:13.396337 28297 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
W20260709 12:15:13.469540 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 74322.4us
I20260709 12:15:13.469937 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.470811 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:13.473899 28314 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:15:13.478853 24015 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:13.452075 27900 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.454200 28117 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:13.468546 28145 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.368552 28298 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.560683 28145 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.562346 28298 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:13.376010 28166 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:15:13.484159 28049 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:13.563271 28049 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:13.564193 28049 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.481581 27900 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:13.480243 27909 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.561s	user 0.013s	sys 0.000s
I20260709 12:15:13.484560 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.397193 28303 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.517125 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:13.534546 28308 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 12:15:13.457283 28302 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:13.551491 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 156377us
I20260709 12:15:13.432960 28286 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.555038 23395 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:13.476167 28083 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.558965 28064 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.558046 23470 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:13.560014 28312 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.103s	user 0.003s	sys 0.000s
I20260709 12:15:13.572106 24012 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:13.581262 23390 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:13.585532 23467 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:13.609827 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.610487 23470 raft_consensus.cc:2376] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11. Re-sending same reply.
I20260709 12:15:13.583992 28303 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:13.620296 23461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.016s	sys 0.020s
W20260709 12:15:13.620712 23461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.016s	sys 0.020s
W20260709 12:15:13.627785 23436 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.016s	sys 0.014s
W20260709 12:15:13.628149 23436 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.017s	sys 0.014s
W20260709 12:15:13.628410 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 557585us
I20260709 12:15:13.631737 28237 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:13.632361 28237 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:13.636716 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:13.638491 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:13.638811 28064 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:13.639081 28189 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:13.639657 28084 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:13.640372 28084 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:13.585346 27909 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.642825 27909 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:13.619743 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 177037us
W20260709 12:15:13.589959 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 217077us
I20260709 12:15:13.581997 24008 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:13.587224 23463 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:13.645128 23392 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:13.629097 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:13.645713 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 173087us
I20260709 12:15:13.648492 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:13.622835 28286 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:13.659420 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 292568us
I20260709 12:15:13.653404 28042 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.662127 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:13.667992 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.664429 28121 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.692631 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:13.691565 28237 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:15:13.705024 28326 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:13.705672 28326 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:13.706799 28326 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.711232 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.711521 27909 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:13.673219 28189 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.712471 28189 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:13.713042 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.714955 28189 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:13.716153 28237 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:13.716284 28117 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.682792 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:13.717980 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.718865 28117 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:13.692073 28321 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:15:13.724133 27900 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.688570 28325 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:13.726861 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.727504 23461 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:13.727921 28325 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:13.728384 28325 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:13.728449 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.728921 28325 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:13.729839 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.736260 23461 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:13.735550 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86568.9us
I20260709 12:15:13.740185 23314 pending_rounds.cc:85] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:13.740753 23314 pending_rounds.cc:107] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:15:13.741148 23314 raft_consensus.cc:2889] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:13.745223 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.746704 27841 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.749811 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:13.755659 24008 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:13.684517 28166 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:13.792085 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:13.792662 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 104334us
I20260709 12:15:13.793099 28311 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:13.801647 28311 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:13.792305 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.794533 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.792409 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.792833 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.793056 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.793900 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:13.798913 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 93134.5us
I20260709 12:15:13.808130 28311 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:13.804248 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 88372.2us
I20260709 12:15:13.811410 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:13.813373 24015 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:13.815435 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:13.817912 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.818518 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.819960 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:13.805923 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9623630579999372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
I20260709 12:15:13.824787 28331 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.825600 28331 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:13.827025 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.828785 28331 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:13.685786 28323 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
W20260709 12:15:13.835551 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 147223us
I20260709 12:15:13.823329 28311 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:13.836489 28316 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:13.846256 27841 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:13.870265 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:13.874006 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 54168.3us
I20260709 12:15:13.875509 23461 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 7: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:13.878643 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 195963us
I20260709 12:15:13.879534 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:13.880921 28303 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:13.881480 28303 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.881836 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:13.881536 28335 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:15:13.883664 28335 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.883885 28303 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:13.885370 28293 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.547s	user 0.005s	sys 0.000s
I20260709 12:15:13.885572 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:13.887024 28329 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:13.887554 28329 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:13.887794 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1003) took 1045 ms. Trace:
I20260709 12:15:13.888540 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.890735 28339 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:13.891042 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:13.892156 28049 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:13.892522 28049 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.892879 28049 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:13.891160 28339 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:13.893748 28339 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.894151 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:13.846768 28311 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:13.884598 28335 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:13.888532 28329 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.921257 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.888336 23464 rpcz_store.cc:276] 0709 12:15:12.842580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.851704 (+  9124us) service_pool.cc:224] Handling call
0709 12:15:12.873106 (+ 21402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:12.887068 (+ 13962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:12.887096 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:12.887595 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:15:13.005205 (+117610us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:13.134512 (+129307us) log.cc:844] Serialized 14 byte log entry
0709 12:15:13.314626 (+180114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:13.314712 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:13.808488 (+493776us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:13.887417 (+ 78929us) raft_consensus.cc:1685] finished
0709 12:15:13.887545 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:13.887717 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11997,"prepare.queue_time_us":253389,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":70055,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":9290,"spinlock_wait_cycles":17566464,"thread_start_us":399093,"threads_started":3,"wal-append.queue_time_us":315216}
I20260709 12:15:13.921377 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.924201 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.925683 23465 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:13.927762 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.934314 23461 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:13.935742 28335 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:13.913560 28311 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:13.936836 28335 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:13.925447 28303 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:13.940711 28303 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:13.928488 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.906677 27841 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:13.941792 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.946079 23463 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:15:13.947752 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53679.6us
I20260709 12:15:13.949023 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:13.951845 28083 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:13.958369 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118422us
I20260709 12:15:13.953188 28083 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:13.963807 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.963321 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:13.901145 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:13.964579 23287 connection.cc:256] RPC call timeout handler was delayed by -1.930764950999901s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 12:15:13.965037 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9271649509998952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 12:15:13.965269 23287 connection.cc:256] RPC call timeout handler was delayed by -1.7801816569999573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20070000000000002s for a response
W20260709 12:15:13.965523 23287 connection.cc:256] RPC call timeout handler was delayed by -1.7139630579999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
W20260709 12:15:13.965762 23287 connection.cc:256] RPC call timeout handler was delayed by -1.5239649509999253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24820000000000003s for a response
W20260709 12:15:13.966339 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:13.965917 23461 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:13.927726 27900 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:13.969955 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 88180us
I20260709 12:15:13.967671 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.971433 23390 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:13.971325 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:13.973006 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.974107 23461 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:13.969904 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:13.978911 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.979643 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:13.979815 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:13.979594 23393 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:13.981065 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:13.982218 23318 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:13.981371 23396 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:13.986490 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.984845 27900 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.985737 28323 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:13.987373 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.988300 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:13.942574 28331 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:15:13.991328 28323 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:13.997046 28323 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:13.997692 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:13.998265 28083 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:13.999982 23465 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:14.002058 28316 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.003173 28334 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:14.003348 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:14.002179 23394 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:14.011368 23465 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:14.012850 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:14.017995 28117 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:14.018453 28117 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:13.998172 28311 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.018913 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:14.021085 28311 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:14.022341 23469 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:15:14.028066 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:14.033049 23461 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:14.035166 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:14.035944 23318 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:14.035249 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:14.008450 27900 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:15:14.042264 24008 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:14.045032 23394 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:14.045807 24011 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:14.014844 28334 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.051442 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:14.054049 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:14.052729 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.043736 28302 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.066572 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:14.069777 23394 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:14.067899 24010 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:14.069643 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:14.073664 23318 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.053464 28334 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.080227 23468 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:14.084017 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:14.086659 28303 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:14.060760 28316 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.088037 28323 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.088900 23316 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:15:14.092633 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76737.6us
I20260709 12:15:14.072607 28302 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:14.094282 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 103718us
I20260709 12:15:14.113919 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:14.101904 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:14.108330 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.093408 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:14.115984 23321 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.061005 28286 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:15:14.120839 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:14.121686 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.114766 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:14.083182 28334 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.127249 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:14.094276 28316 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.048931 27841 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.127452 23316 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:14.130614 23396 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:14.115365 28302 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.134796 28323 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:14.138027 23387 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:14.123471 28303 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:14.138078 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:14.142468 23467 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:15:14.093998 28311 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:15:14.149137 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:14.145653 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:14.150063 24008 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.130735 28316 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.157703 28339 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.127157 28335 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:15:14.178860 28358 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:14.179697 28358 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:14.180276 28042 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.517s	user 0.004s	sys 0.007s
I20260709 12:15:14.150394 28299 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:15:14.181792 28299 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:14.181947 28329 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:14.182224 28299 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:14.182578 28329 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.183086 28329 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.183187 28299 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.184464 28299 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.150321 28350 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:14.186012 28350 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:14.186993 28350 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.139179 28342 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:15:14.168939 28339 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:14.169902 28348 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
W20260709 12:15:14.172111 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.172457 28064 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:14.173044 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 202972us
I20260709 12:15:13.902557 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:14.189400 23466 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:14.142118 28303 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:14.258844 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 865) took 1093 ms. Trace:
I20260709 12:15:14.266417 23318 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:14.263087 23392 rpcz_store.cc:276] 0709 12:15:13.165553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.165666 (+   113us) service_pool.cc:224] Handling call
0709 12:15:14.258637 (+1092971us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54002c7754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:14.258785 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079383936}
I20260709 12:15:14.279911 23318 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:15:14.263434 28303 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:14.283540 23321 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:14.165287 28298 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:15:14.294754 24008 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
W20260709 12:15:14.291003 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 161216us
I20260709 12:15:14.293421 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:14.293999 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1014) took 1141 ms. Trace:
I20260709 12:15:14.295883 24015 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:14.295802 23462 rpcz_store.cc:276] 0709 12:15:13.152755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.336474 (+183719us) service_pool.cc:224] Handling call
0709 12:15:13.340462 (+  3988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:13.584289 (+243827us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:13.584675 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 12:15:13.672304 (+ 87629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:13.672357 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:13.681469 (+  9112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.293773 (+612304us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54000c7454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:14.293828 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.293946 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2069,"prepare.queue_time_us":995,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":3130,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":38111,"spinlock_wait_cycles":560050560,"thread_start_us":85107,"threads_started":1,"wal-append.queue_time_us":148980}
I20260709 12:15:14.294871 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:14.303414 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 165404us
I20260709 12:15:14.180908 28358 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:14.304922 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 185098us
I20260709 12:15:14.279328 28341 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.261137 28113 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:15:14.263599 28064 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.311520 23392 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:14.320906 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:14.326314 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:14.332641 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56683.1us
I20260709 12:15:14.300004 28237 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 12:15:14.391841 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.402002 24015 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:14.323042 28342 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:15:14.406405 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:14.324765 28323 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:15:14.353008 28286 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:14.408012 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:14.325542 28049 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.331698 28341 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.411473 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:14.413898 23319 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:14.407356 28309 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.764s	user 0.011s	sys 0.000s
I20260709 12:15:14.358099 28042 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:15:14.401641 28294 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:14.419287 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 94050.1us
I20260709 12:15:14.424213 24009 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:14.425788 28319 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.788s	user 0.012s	sys 0.005s
I20260709 12:15:14.374015 28359 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:15:14.427652 28335 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.416805 27841 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:15:14.486557 23312 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:14.488358 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:14.495033 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 164419us
I20260709 12:15:14.456110 28237 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:14.430574 28336 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.232s
W20260709 12:15:14.540572 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 132648us
I20260709 12:15:14.541642 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:14.485872 28335 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:14.428109 28294 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:14.541373 28319 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:14.568035 28335 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.562996 28301 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:15:14.645881 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 223109us
I20260709 12:15:14.602097 28237 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:14.661012 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:14.691907 28370 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:15:14.693917 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2363) took 1486 ms. Trace:
I20260709 12:15:14.695151 23464 rpcz_store.cc:276] 0709 12:15:13.207100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.988985 (+781885us) service_pool.cc:224] Handling call
0709 12:15:14.147973 (+158988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:14.152036 (+  4063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:14.152066 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:14.162695 (+ 10629us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:14.162736 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.162800 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:14.693661 (+530861us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54001879d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:14.693837 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349002624}
I20260709 12:15:14.698298 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1013) took 1549 ms. Trace:
I20260709 12:15:14.698846 23469 rpcz_store.cc:276] 0709 12:15:13.149352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.304427 (+155075us) service_pool.cc:224] Handling call
0709 12:15:13.304787 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:14.023372 (+718585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:14.697792 (+674420us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b5400186fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:14.698178 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649558272}
W20260709 12:15:14.706259 28367 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:15:14.713917 28118 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:14.714416 28118 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:14.715306 28118 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.716579 28118 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:14.725059 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:14.721517 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 398911us
I20260709 12:15:14.727733 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:14.635583 28301 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:14.670074 28064 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:14.748293 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:14.749171 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 207604us
I20260709 12:15:14.753337 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:14.757444 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:14.685499 28342 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:14.739485 28049 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:14.769584 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 84465.2us
I20260709 12:15:14.749928 28113 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:14.741914 28301 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:14.782143 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:14.782387 28378 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:15:14.783151 28117 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.784343 28113 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:14.784428 28117 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:14.786906 28364 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:15:14.783936 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.790473 28113 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:14.790963 28113 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:14.791380 28113 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:14.796243 23287 leader_election.cc:400] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:14.798861 28350 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:14.799521 28350 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.758760 28357 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:14.799508 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:14.757033 28325 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.028s	user 0.016s	sys 0.000s
W20260709 12:15:14.815387 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:14.815481 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.759929 27909 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:14.840186 28166 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:14.807150 28145 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:15:14.813725 28380 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:15:14.805354 28329 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.000s
I20260709 12:15:14.841234 28064 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:14.841846 28357 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:14.765476 28342 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:14.906243 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:14.847407 28321 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:15:14.848610 28316 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.003s
I20260709 12:15:14.849421 28386 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:14.850281 28382 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:14.858129 24012 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:14.859127 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2369) took 1383 ms (client timeout 1590 ms). Trace:
I20260709 12:15:14.860823 24015 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:14.866137 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:14.817866 28365 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.276s	user 0.003s	sys 0.000s
W20260709 12:15:14.927589 23468 rpcz_store.cc:269] 0709 12:15:13.475839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.516747 (+ 40908us) service_pool.cc:224] Handling call
0709 12:15:14.858845 (+1342098us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54000e1d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:14.859038 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":862689152}
W20260709 12:15:14.805354 28302 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 12:15:14.821681 28326 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:14.806378 28341 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:14.912849 28387 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:14.922166 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.815598 28351 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:15:14.926187 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:14.930675 23319 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.001s
I20260709 12:15:14.884090 28064 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:14.831295 28325 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.881358 28166 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.805752 28237 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.960958 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:14.922463 28376 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:15:14.985613 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 204495us
I20260709 12:15:15.002351 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:14.905712 28329 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:14.973045 28334 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.832s	user 0.013s	sys 0.000s
I20260709 12:15:14.964784 23319 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:14.966184 28145 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:14.973021 28064 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:15.013025 23980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.015s	sys 0.035s
W20260709 12:15:15.024188 23980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.015s	sys 0.035s
I20260709 12:15:15.024892 23396 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:15.024797 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 598411us
I20260709 12:15:14.992915 28335 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:14.973907 28326 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:14.982987 28387 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:14.927850 28316 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.973309 28351 consensus_queue.cc:1243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:15.035393 23387 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:15.044929 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.057359 23468 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:14.824440 28301 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:14.939473 28302 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.011816 28376 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:15.064627 23434 connection.cc:256] RPC call timeout handler was delayed by -1.0468269710002005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2911s for a response
I20260709 12:15:15.013877 28145 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:15.065235 23467 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [11.8-12.9]
I20260709 12:15:15.072031 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.068347 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.075193 28145 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:15.076154 28325 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.077265 23396 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:15.081379 23466 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:15.086182 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 120096us
I20260709 12:15:15.051816 28042 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:15.089623 23461 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:15.060513 27909 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.089350 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:15.093907 23465 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:15.097249 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.015739 28334 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.099619 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:15.097146 28303 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.796s	user 0.011s	sys 0.002s
I20260709 12:15:15.068001 28084 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:15:15.062575 28113 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:15.115054 23206 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:15:15.115366 28340 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:15.019073 28342 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:15.104241 28299 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.919s	user 0.015s	sys 0.002s
I20260709 12:15:15.106637 28358 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.111265 28343 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:15.130892 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43592.8us
W20260709 12:15:15.129961 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.124222 28402 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:15.066668 28357 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.141855 28166 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.142390 24016 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:15:15.143075 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.145280 28316 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.145817 28386 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.147121 27909 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.148303 28145 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.149727 28298 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.149612 28326 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.150368 28298 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.150897 28113 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.152537 28298 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:15.141726 28375 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:15:15.110493 28042 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.165455 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:15.171204 24008 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.119596 28340 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:15.173574 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:15.140642 28342 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:15.128647 28325 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:15:15.117309 28303 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.179140 28400 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:15.182286 28118 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:15.162822 28316 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:15.188381 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:15.195170 24008 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:15.146401 28386 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.202826 28335 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.203749 28340 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.204618 28386 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.205073 28340 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:15.138368 28376 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.207470 28064 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.190086 28118 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:15.210129 28118 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.214841 28042 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.216832 28145 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.138836 28358 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:15.220583 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.223925 23469 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:15.191425 28400 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.235102 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.237643 28402 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:15.238296 28402 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:15.239248 28408 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:15:15.163803 28387 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.223564 28294 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.199049 28396 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:15:15.200110 28329 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:15.247191 28302 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.247910 28302 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.248220 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:15.248446 28302 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.249701 28301 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.252849 28334 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.241816 24012 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:15.257043 28402 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:15.257798 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.272091 28402 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:15.278717 28402 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.279937 28303 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.281423 28402 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.283569 23319 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:15.234550 28400 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.219384 28316 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.281144 28118 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.293828 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.295413 28118 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.295975 28316 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.299084 28302 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.301364 28358 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.301901 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:15.302867 24012 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:15.293165 28334 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:15:15.281889 28064 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.309875 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.310513 23462 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.311998 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.312625 23464 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:15.314066 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:15.312028 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.290925 23313 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:15:15.292676 23314 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:15.324445 27900 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:15.324982 27900 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:15.325039 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.325479 27900 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:15.326318 28363 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.870s	user 0.002s	sys 0.002s
I20260709 12:15:15.322009 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:15.328544 23312 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.341660 24015 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:15.342423 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2373) took 1806 ms. Trace:
I20260709 12:15:15.343039 23461 rpcz_store.cc:276] 0709 12:15:13.536345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.101662 (+565317us) service_pool.cc:224] Handling call
0709 12:15:14.101960 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.090040 (+988080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.342039 (+251999us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54003da754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:15.342252 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2270939008}
I20260709 12:15:15.344012 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.344722 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.344741 23464 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.347393 28323 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:15.347859 28323 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.354043 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.345388 23461 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16. Re-sending same reply.
I20260709 12:15:15.377812 23462 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:15.337105 28350 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.365121 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:15.385962 23312 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:15.386047 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:15.387812 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.392208 23391 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 12:15:15.450527 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.387822 24015 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:15.450112 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:15.458678 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.457095 24011 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:15.451733 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:15.452888 24012 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:15.451256 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:15.451521 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 89754.4us
I20260709 12:15:15.458254 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.463450 23390 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:15.384896 28316 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.389899 27909 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:15.467401 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:15.485323 23319 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 16 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:15:15.470352 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.349151 28323 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:15.487090 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 382776us
I20260709 12:15:15.487272 23464 raft_consensus.cc:2376] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12. Re-sending same reply.
I20260709 12:15:15.487591 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:15.489665 23317 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:15.366192 28386 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:15:15.373296 27841 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:15:15.506963 23396 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.501933 23321 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:15.492311 23314 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:15.463538 28339 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.466166 28374 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:15.513514 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:15.501621 28117 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.717s	user 0.016s	sys 0.001s
I20260709 12:15:15.544243 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:15.513815 28343 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:15:15.504181 28316 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.492887 28286 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:15.515689 28351 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:15.507067 28303 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:15:15.542037 28323 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:15.541610 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.524591 24007 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:15:15.585182 24007 pending_rounds.cc:85] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 10
I20260709 12:15:15.585503 24007 pending_rounds.cc:107] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:15:15.585772 24007 raft_consensus.cc:2889] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:15.533119 23387 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:15:15.543772 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.537879 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:15.542011 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:15.547628 23461 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:15.517706 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:15.545147 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:15.552923 28049 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:15.554535 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:15.523313 28377 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:15:15.529253 28341 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:15.560916 23319 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:15:15.564769 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:15.524014 28409 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:15:15.576712 23392 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:15:15.620916 23392 pending_rounds.cc:85] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 10
I20260709 12:15:15.621574 23392 pending_rounds.cc:107] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:15:15.622046 23392 raft_consensus.cc:2889] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:15.581772 23396 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:15.590899 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:15.592268 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 74736.2us
I20260709 12:15:15.594884 28113 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:15.596231 28350 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:15.596845 28316 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:15.597864 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1135) took 1452 ms. Trace:
I20260709 12:15:15.568383 28400 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.597635 28351 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:15.519973 27900 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.545169 28339 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.623500 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:15.633497 23434 leader_election.cc:336] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.698s (SENT)
I20260709 12:15:15.650574 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:15.604686 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.639981 24016 rpcz_store.cc:276] 0709 12:15:14.144975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.152195 (+  7220us) service_pool.cc:224] Handling call
0709 12:15:14.255548 (+103353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.142831 (+887283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.597594 (+454763us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b54003741d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:15.597778 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1980919168}
I20260709 12:15:15.623334 23469 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:15.589323 28286 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:15.590684 28049 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:15.659403 23320 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:15.650025 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:15.609628 28341 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.544384 28374 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:15.664456 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.665503 23464 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:15.640017 28357 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:15.695611 23466 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:15.695744 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 153909us
I20260709 12:15:15.647347 28396 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:15.661746 28237 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:15:15.694950 28395 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:15:15.710594 23466 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:15.716187 23320 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:15.717134 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:15.732745 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 82818.2us
I20260709 12:15:15.690562 28166 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:15:15.734601 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:15.736195 23464 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:15.694267 28382 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:15:15.730684 28113 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:15.747625 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:15.779369 28359 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:15.780310 28359 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.750931 28351 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.769330 28341 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.783576 28341 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.785842 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:15.786594 23465 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:15.708859 28396 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.790225 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:15.792363 27841 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:15.763325 28113 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:15.772672 28412 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:15.746469 28329 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:15:15.808118 28064 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:15.840214 28064 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:15.840615 28064 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:15.808380 28113 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:15.738617 28049 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.810731 28384 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:15.873667 27900 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.876812 28384 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.878159 28055 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:15.877913 28384 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.878600 28055 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:15.880471 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.881866 23321 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:15.883074 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.884043 23321 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:15.886534 28049 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:15.887993 28414 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.357s	user 0.003s	sys 0.000s
I20260709 12:15:15.858938 28335 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:15.889020 28335 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.890041 28335 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:15.867403 28420 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:15:15.895323 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 160899us
I20260709 12:15:15.900101 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.903683 23319 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:15.849004 28064 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:15.777959 28286 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.919723 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.921558 23319 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:15.930550 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:15.931576 23319 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:15.911355 28400 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:15.902751 28357 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:15.956218 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.968335 28064 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.968066 28357 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:15.968565 23319 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:15.988914 28407 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:15.989233 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1016) took 2809 ms. Trace:
I20260709 12:15:15.966466 28374 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:15.992336 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1023) took 1279 ms. Trace:
W20260709 12:15:15.991706 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:15.989420 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:15.999109 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1133) took 1837 ms. Trace:
I20260709 12:15:15.989667 23470 rpcz_store.cc:276] 0709 12:15:13.179773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:13.617489 (+437716us) service_pool.cc:224] Handling call
0709 12:15:13.624772 (+  7283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:13.659666 (+ 34894us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:13.670217 (+ 10551us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.091256 (+1421039us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18002cf8c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:15:15.541371 (+450115us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b18002cf8c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:15.601357 (+ 59986us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18002cf8c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0ad06c3c7 00007ff0a967e9a7 00007ff0a968e0bf 00007ff0a968e038
0709 12:15:15.601627 (+   270us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.988553 (+386926us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b18002cf8c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:15.988764 (+   211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:15.988825 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:15.988912 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.988980 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:15.989131 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37140,"prepare.queue_time_us":465363,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":1011,"raft.queue_time_us":407106,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":10614,"spinlock_wait_cycles":94603904,"thread_start_us":41625,"threads_started":1,"wal-append.queue_time_us":1492329}
I20260709 12:15:15.999536 24010 rpcz_store.cc:276] 0709 12:15:14.139195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.139352 (+   157us) service_pool.cc:224] Handling call
0709 12:15:15.975811 (+1836459us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b54006def54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:15.976026 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079795072}
I20260709 12:15:15.994905 23463 rpcz_store.cc:276] 0709 12:15:14.713499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.713645 (+   146us) service_pool.cc:224] Handling call
0709 12:15:14.714026 (+   381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:14.714501 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:14.825230 (+110729us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.162844 (+337614us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:15.163576 (+   732us) log.cc:844] Serialized 52 byte log entry
0709 12:15:15.166753 (+  3177us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:15.636162 (+469409us) log.cc:844] Serialized 14 byte log entry
0709 12:15:15.992034 (+355872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.992105 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:15.992149 (+    44us) raft_consensus.cc:1685] finished
0709 12:15:15.992162 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:15.992289 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":826169,"prepare.queue_time_us":712096,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4571,"raft.queue_time_us":38688,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":240,"thread_start_us":421245,"threads_started":4,"wal-append.queue_time_us":257253}
W20260709 12:15:15.990124 28428 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:15.989530 28407 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:16.024885 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 360024us
I20260709 12:15:16.026098 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:16.026840 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.061088 23319 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.075804 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:16.078378 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:16.086134 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:16.087220 23396 raft_consensus.cc:2368] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:16.088181 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:16.094915 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:16.095278 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.132478 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:16.022801 28359 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:16.142724 23317 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.044718 28237 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:16.151396 23395 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:16.157743 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.158812 23319 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:16.158964 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 547271us
W20260709 12:15:16.165815 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 460752us
I20260709 12:15:16.173002 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:16.179864 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2387) took 1086 ms. Trace:
W20260709 12:15:16.184499 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.201825 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.181062 23466 rpcz_store.cc:276] 0709 12:15:15.092987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.713847 (+620860us) service_pool.cc:224] Handling call
0709 12:15:15.714262 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.714531 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:15.732458 (+ 17927us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.163634 (+431176us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.179153 (+ 15519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.179223 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.179513 (+   290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.179603 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.179779 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":247463,"prepare.queue_time_us":21594,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":4052,"raft.queue_time_us":348756,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8279,"thread_start_us":197888,"threads_started":2,"wal-append.queue_time_us":173162}
W20260709 12:15:16.221673 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.170773 28311 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.232792 23466 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:16.233354 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:16.236739 28439 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.088s	user 0.000s	sys 0.002s
W20260709 12:15:16.238018 28410 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.716s	user 0.003s	sys 0.001s
W20260709 12:15:16.231807 28428 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:15:16.242498 28413 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.713s	user 0.002s	sys 0.008s
I20260709 12:15:16.245404 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2554) took 1924 ms. Trace:
I20260709 12:15:16.245873 23318 rpcz_store.cc:276] 0709 12:15:14.320924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.539943 (+219019us) service_pool.cc:224] Handling call
0709 12:15:15.306143 (+766200us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5800003398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:15:15.306371 (+   228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.306784 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.306813 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.307324 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.372292 (+ 64968us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:15.372333 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.372401 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.241869 (+869468us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:16.245128 (+  3259us) raft_consensus.cc:1685] finished
0709 12:15:16.245156 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.245330 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":618,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":74082816,"thread_start_us":64461,"threads_started":1,"wal-append.queue_time_us":200825}
I20260709 12:15:16.250682 28339 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:16.186012 28298 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:16.266441 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:16.267145 23466 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:15:16.271649 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.276674 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.280738 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:16.281629 24008 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:16.290668 24007 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:16.358450 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.374656 28440 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:16.375486 28438 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:15:16.375329 28440 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:16.377815 28436 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.287s	user 0.000s	sys 0.003s
I20260709 12:15:16.168351 28407 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.380395 28449 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:16.382028 28444 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:15:16.383487 28448 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:16.384310 28448 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.381554 28449 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.389205 28449 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.389810 28449 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.390789 28455 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:16.390480 28449 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.265583 28298 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.393461 28298 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.394606 28298 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.394738 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.395886 23319 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.397017 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:16.397388 28323 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.807s	user 0.011s	sys 0.001s
I20260709 12:15:16.397706 23319 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:16.260084 28237 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:16.398609 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.399361 23319 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.400130 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.400689 23319 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:15:16.394933 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.300653 28404 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:16.402418 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.255470 28311 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.393500 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.409474 23317 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:16.409698 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.410928 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:16.411798 23466 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:16.411792 23312 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.418181 24008 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:16.422055 23312 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:16.424593 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.425661 23312 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:16.432723 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.434476 23318 raft_consensus.cc:2410] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:15:16.439375 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.439899 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.440447 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2400) took 1081 ms. Trace:
I20260709 12:15:16.443914 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.445305 23317 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:15:16.441002 23467 rpcz_store.cc:276] 0709 12:15:15.358928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.156901 (+797973us) service_pool.cc:224] Handling call
0709 12:15:16.169005 (+ 12104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.169278 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:16.184836 (+ 15558us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.430226 (+245390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.430288 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:16.430344 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.430416 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.440288 (+  9872us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42708,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":231894,"threads_started":1}
I20260709 12:15:16.456969 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2382) took 1407 ms. Trace:
I20260709 12:15:16.395233 28298 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.467629 23319 raft_consensus.cc:2410] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:16.428828 28339 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.442894 28448 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.459496 23462 rpcz_store.cc:276] 0709 12:15:15.049609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.655611 (+606002us) service_pool.cc:224] Handling call
0709 12:15:15.656292 (+   681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.656598 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:15.697595 (+ 40997us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.453792 (+756197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.455630 (+  1838us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:16.456632 (+  1002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.456704 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.456835 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":220539,"prepare.queue_time_us":44950,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":357389,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":5882,"thread_start_us":534821,"threads_started":2,"wal-append.queue_time_us":222561}
I20260709 12:15:16.471346 23214 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:16.480638 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.453469 28118 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.006s	sys 0.000s
I20260709 12:15:16.496496 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:16.497658 23317 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.391325 28455 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.469359 28400 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.530192 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1032) took 1010 ms. Trace:
I20260709 12:15:16.530707 23467 rpcz_store.cc:276] 0709 12:15:15.520201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.490072 (+969871us) service_pool.cc:224] Handling call
0709 12:15:16.529699 (+ 39627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.529874 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.529899 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.529937 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:16.529964 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.530012 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.530137 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:16.535583 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2380) took 1486 ms. Trace:
I20260709 12:15:16.538218 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2381) took 1489 ms. Trace:
I20260709 12:15:16.538682 23469 rpcz_store.cc:276] 0709 12:15:15.049280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.652253 (+602973us) service_pool.cc:224] Handling call
0709 12:15:15.690040 (+ 37787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:16.089980 (+399940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.090011 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.091929 (+  1918us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.231043 (+139114us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.231178 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.231370 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.537936 (+306566us) raft_consensus.cc:1685] finished
0709 12:15:16.537961 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.538149 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":365056,"raft.queue_time_us":371385,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11077,"thread_start_us":130465,"threads_started":2,"wal-append.queue_time_us":336926}
I20260709 12:15:16.542213 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.543190 23319 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:16.543679 28415 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.025s	user 0.003s	sys 0.000s
I20260709 12:15:16.545466 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1290) took 1256 ms. Trace:
I20260709 12:15:16.546444 23316 rpcz_store.cc:276] 0709 12:15:15.289548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.289722 (+   174us) service_pool.cc:224] Handling call
0709 12:15:15.325779 (+ 36057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:15.326227 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:15.326260 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:15.326784 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:15:15.428397 (+101613us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:15.428461 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:15.428574 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.483737 (+1055163us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:16.545117 (+ 61380us) raft_consensus.cc:1685] finished
0709 12:15:16.545139 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.545387 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1232,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":53854592,"thread_start_us":101127,"threads_started":1,"wal-append.queue_time_us":191633}
I20260709 12:15:16.555985 28373 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:16.556596 28373 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:16.563659 28449 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.564682 28400 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:16.486068 28335 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:16.566112 28335 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.566969 28400 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:16.568198 28335 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.569720 28311 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:16.570653 28113 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.001s
I20260709 12:15:16.571066 28113 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:16.572119 28441 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.273s	user 0.000s	sys 0.002s
I20260709 12:15:16.572225 23320 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:16.577646 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2389) took 1448 ms. Trace:
I20260709 12:15:16.589012 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1033) took 1068 ms. Trace:
I20260709 12:15:16.589705 23469 rpcz_store.cc:276] 0709 12:15:15.520898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.540575 (+1019677us) service_pool.cc:224] Handling call
0709 12:15:16.540965 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.541202 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.541236 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.541304 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:16.541347 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.541415 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.588936 (+ 47521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:16.592545 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:16.523957 28118 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.006s	sys 0.001s
I20260709 12:15:16.536952 28298 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.536196 23461 rpcz_store.cc:276] 0709 12:15:15.048949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.632845 (+583896us) service_pool.cc:224] Handling call
0709 12:15:15.653353 (+ 20508us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:16.023375 (+370022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.023407 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:16.086344 (+ 62937us) log.cc:844] Serialized 52 byte log entry
0709 12:15:16.238191 (+151847us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b480021c258. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:15:16.265529 (+ 27338us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.355357 (+ 89828us) log.cc:844] Serialized 14 byte log entry
0709 12:15:16.355650 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.355767 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.535154 (+179387us) raft_consensus.cc:1685] finished
0709 12:15:16.535287 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.535492 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341593,"prepare.queue_time_us":90164,"prepare.run_cpu_time_us":2632,"prepare.run_wall_time_us":18335,"raft.queue_time_us":383269,"raft.run_cpu_time_us":2429,"raft.run_wall_time_us":176811,"spinlock_wait_cycles":10947712,"thread_start_us":106964,"threads_started":1,"wal-append.queue_time_us":334568}
I20260709 12:15:16.603420 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.578229 23465 rpcz_store.cc:276] 0709 12:15:15.105232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.787308 (+682076us) service_pool.cc:224] Handling call
0709 12:15:15.788817 (+  1509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:16.155666 (+366849us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:15:16.155876 (+   210us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.167553 (+ 11677us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.410865 (+243312us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:16.410903 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.410970 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.553531 (+142561us) raft_consensus.cc:1685] finished
0709 12:15:16.553552 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.553732 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":229936,"raft.queue_time_us":361017,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":6174,"thread_start_us":309623,"threads_started":2,"wal-append.queue_time_us":337050}
I20260709 12:15:16.606766 28357 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:15:16.609938 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1035) took 1088 ms. Trace:
I20260709 12:15:16.610349 23461 rpcz_store.cc:276] 0709 12:15:15.521877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.606586 (+1084709us) service_pool.cc:224] Handling call
0709 12:15:16.609445 (+  2859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.609628 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.609652 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.609691 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:16.609719 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.609777 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.609882 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:16.611259 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.611792 23461 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:15:16.612001 23316 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:16.613148 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:16.604488 28113 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.615689 28376 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:16.617723 28376 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:16.618685 28376 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:16.623993 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1038) took 1031 ms. Trace:
I20260709 12:15:16.627568 23465 rpcz_store.cc:276] 0709 12:15:15.592388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.608617 (+1016229us) service_pool.cc:224] Handling call
0709 12:15:16.623338 (+ 14721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.623565 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.623605 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.623663 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:16.623708 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.623777 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.623914 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:16.487829 28339 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.634446 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:16.635604 23467 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:16.636785 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1036) took 1114 ms. Trace:
I20260709 12:15:16.637755 28437 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:16.638325 28437 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:16.639019 24008 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:16.640750 24007 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:16.639158 28437 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.637355 23467 rpcz_store.cc:276] 0709 12:15:15.522163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.607217 (+1085054us) service_pool.cc:224] Handling call
0709 12:15:16.634738 (+ 27521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.636399 (+  1661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.636710 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:16.652158 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:16.632925 28341 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:16.662513 23470 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.000s	sys 0.015s
W20260709 12:15:16.665757 23470 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.000s	sys 0.016s
I20260709 12:15:16.660676 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.673120 23316 raft_consensus.cc:2410] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:15:16.674227 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.655012 28049 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.764s	user 0.013s	sys 0.000s
I20260709 12:15:16.648137 28298 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:16.662860 28393 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:16.597407 28373 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.693075 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.665827 28341 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:16.694128 24007 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:16.701922 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.707800 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:16.699012 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.703651 28373 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:16.730728 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.731964 23319 raft_consensus.cc:2410] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:15:16.724223 28335 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:16.737916 28401 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.196s	user 0.000s	sys 0.007s
I20260709 12:15:16.739612 28339 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:16.714792 28341 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.741533 28341 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:16.691044 28298 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:16.695782 28393 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.752517 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.752978 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.753176 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.753803 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:16.753870 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.754470 23318 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.754601 23321 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:16.755205 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2388) took 1661 ms. Trace:
I20260709 12:15:16.755724 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.756053 23316 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:16.756533 23312 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.757826 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.757210 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:16.755733 23464 rpcz_store.cc:276] 0709 12:15:15.093294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.739234 (+645940us) service_pool.cc:224] Handling call
0709 12:15:15.765103 (+ 25869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:16.452510 (+687407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.452569 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:16.479048 (+ 26479us) log.cc:844] Serialized 26 byte log entry
0709 12:15:16.723709 (+244661us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.723759 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.723854 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.754908 (+ 31054us) raft_consensus.cc:1685] finished
0709 12:15:16.754949 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.755130 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251144,"raft.queue_time_us":367843,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":9709,"thread_start_us":617892,"threads_started":2,"wal-append.queue_time_us":257694}
I20260709 12:15:16.760207 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.762703 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.754122 28357 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:15:16.759811 28339 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.774667 28393 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.779191 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.779445 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.780157 23317 raft_consensus.cc:2410] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:16.780692 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.780890 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:16.781785 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:16.788975 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.789719 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:16.791968 28440 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:16.754395 28384 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:15:16.762163 28326 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:16.796761 28443 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.565s	user 0.001s	sys 0.001s
I20260709 12:15:16.798084 28339 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.799027 28298 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:16.793092 28084 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:15:16.812722 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.820277 23316 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:16.813665 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.817020 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.806629 28113 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:16.820937 23319 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:15:16.821919 23321 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
W20260709 12:15:16.824677 28412 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.000s	sys 0.010s
I20260709 12:15:16.812291 28326 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:16.829694 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:16.830586 24010 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:16.831385 28326 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.833318 28055 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.834800 28055 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:16.819907 28084 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:16.851881 24008 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:15:16.808573 28357 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
W20260709 12:15:16.828748 28412 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.000s	sys 0.011s
I20260709 12:15:16.850870 28326 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.855578 28339 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:16.860961 28326 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:16.861518 28412 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:16.862864 28412 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:16.864678 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:16.865686 28339 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:16.856150 28237 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:16.866183 28339 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.858481 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2394) took 1611 ms. Trace:
I20260709 12:15:16.867395 23470 rpcz_store.cc:276] 0709 12:15:15.247237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.002458 (+755221us) service_pool.cc:224] Handling call
0709 12:15:16.016421 (+ 13963us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:16.666247 (+649826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.666278 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:16.687333 (+ 21055us) log.cc:844] Serialized 52 byte log entry
0709 12:15:16.806747 (+119414us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:16.806786 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.806862 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:16.858239 (+ 51377us) raft_consensus.cc:1685] finished
0709 12:15:16.858261 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.858423 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111286,"raft.queue_time_us":212317,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":17532,"thread_start_us":633733,"threads_started":2,"wal-append.queue_time_us":131140}
I20260709 12:15:16.869552 28084 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:16.871088 28084 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:16.816250 28440 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 12:15:16.876914 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.878962 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:16.882524 23387 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 877) took 1794 ms (client timeout 1881 ms). Trace:
W20260709 12:15:16.883083 23387 rpcz_store.cc:269] 0709 12:15:15.087719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.087845 (+   126us) service_pool.cc:224] Handling call
0709 12:15:16.882283 (+1794438us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54001e3f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:16.882413 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":991277824}
I20260709 12:15:16.886691 23467 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:15:16.857533 28425 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.005s	sys 0.020s
I20260709 12:15:16.886691 23464 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:16.898676 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:16.900141 23470 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:16.901901 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:16.902668 23465 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:16.887341 28358 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:15:16.888502 28425 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.005s	sys 0.021s
I20260709 12:15:16.911966 23464 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:16.921701 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:16.922603 23470 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:16.927591 23321 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 6. Current term is 7. Ops: []
I20260709 12:15:16.929023 23321 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:16.929800 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.930040 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:16.930536 23319 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:16.933857 23314 raft_consensus.cc:2410] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:15:16.931170 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1027) took 1558 ms. Trace:
I20260709 12:15:16.931201 23318 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:16.936452 23463 rpcz_store.cc:276] 0709 12:15:15.372871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.165663 (+792792us) service_pool.cc:224] Handling call
0709 12:15:16.179132 (+ 13469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.405017 (+225885us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.405064 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.405121 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.405167 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.930819 (+525652us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54000c8354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:16.930975 (+   156us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:16.931116 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518850432}
I20260709 12:15:16.941385 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:16.934185 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 871) took 2021 ms. Trace:
I20260709 12:15:16.943287 23392 rpcz_store.cc:276] 0709 12:15:14.912822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:14.926937 (+ 14115us) service_pool.cc:224] Handling call
0709 12:15:14.958383 (+ 31446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:15.647194 (+688811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:16.933961 (+1286767us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:16.934126 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420692224}
W20260709 12:15:16.953780 28351 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.005s	sys 0.016s
I20260709 12:15:16.955521 23470 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
W20260709 12:15:17.051739 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86670.2us
W20260709 12:15:16.958498 28351 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.173s	user 0.005s	sys 0.016s
I20260709 12:15:17.070209 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:16.950924 28334 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:16.917888 28440 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.138s
W20260709 12:15:17.054864 28474 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:15:17.084918 23436 leader_election.cc:400] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:17.085340 23436 leader_election.cc:403] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:17.058674 28382 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:15:16.968781 28477 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:17.097886 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 186137us
I20260709 12:15:17.061450 28391 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:17.086858 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 170851us
I20260709 12:15:17.124166 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:17.090456 28303 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:17.126282 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 73735.4us
I20260709 12:15:17.111009 28450 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:17.102466 28477 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:17.132335 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:17.088312 28334 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.125389 28339 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:17.154743 28117 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.157755 28117 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:17.158491 28481 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:15:17.160207 28117 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:17.161159 28334 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:17.161732 28341 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:17.162271 28489 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:17.162842 28489 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.163203 28489 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.170341 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.171931 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.172842 28113 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.176604 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.122764 28391 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.178964 28326 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:17.179343 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.179788 28326 consensus_queue.cc:1243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:15:17.181809 28440 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:17.182456 28440 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:17.173632 28113 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.132019 28477 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:17.192127 28479 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:15:17.210925 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.187558 28339 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:17.237670 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.238668 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.216696 28412 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.237339 28113 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:15:17.220943 28323 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:17.230947 28339 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:17.259327 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:17.271404 28339 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.197799 28391 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:17.271713 28323 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.304766 28323 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:17.356918 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 146127us
I20260709 12:15:17.357757 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:17.272787 28440 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.368757 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.371025 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.372457 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.361956 28501 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:17.378222 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.378875 23470 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:17.379477 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.373579 28412 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:17.381671 28412 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:17.383981 23387 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:17.384644 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.364711 28237 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:15:17.372121 28113 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.371531 28503 raft_consensus.cc:2764] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:15:17.384579 28501 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.414244 28113 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:17.440943 23464 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:15:17.442157 28497 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:17.442998 23464 pending_rounds.cc:85] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 5
I20260709 12:15:17.443572 23464 pending_rounds.cc:107] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:15:17.444053 23464 raft_consensus.cc:2889] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:17.441231 28339 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.411149 28412 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:17.449843 28339 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:17.446734 28489 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:17.455845 28502 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:15:17.457360 28396 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.452345 28477 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:17.465358 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.482960 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.483973 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1297) took 1903 ms. Trace:
I20260709 12:15:17.484344 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.484408 23320 rpcz_store.cc:276] 0709 12:15:15.580257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:15.740590 (+160333us) service_pool.cc:224] Handling call
0709 12:15:15.762634 (+ 22044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.573015 (+810381us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.483723 (+910708us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:17.483910 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861323136}
I20260709 12:15:17.485564 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.486759 24008 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:17.488709 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.489841 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:17.490406 28491 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:15:17.491086 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.491832 23470 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:17.492715 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.494234 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.494657 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.496366 23467 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:17.503758 24008 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:17.475663 28487 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.506665 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.507336 23389 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:17.507441 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.508021 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:17.508502 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 136133us
I20260709 12:15:17.513197 23394 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:15:17.516382 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.519390 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.522063 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.525895 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.528445 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.525895 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:17.558621 28385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.684s	user 0.013s	sys 0.006s
W20260709 12:15:17.559321 28385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.685s	user 0.013s	sys 0.006s
I20260709 12:15:17.559975 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:17.563220 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.564410 23470 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:17.564334 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.478724 28465 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:15:17.568109 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.566731 23395 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:15:17.566498 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:17.525923 28447 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.298s	user 0.001s	sys 0.004s
I20260709 12:15:17.518396 28501 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.570288 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.574766 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.576684 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.570480 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.579234 24007 raft_consensus.cc:2368] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:17.567925 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.562493 28412 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:17.581630 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:17.595914 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:17.602243 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.610519 28489 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:17.611651 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.605332 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.614506 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.617884 24008 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:17.511854 28487 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.613838 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.620117 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.621156 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.626192 24012 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:17.623745 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.621915 28373 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:15:17.630426 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.631245 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:17.631610 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.631950 24007 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:17.632784 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.643410 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.588127 28385 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:17.629527 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:17.645625 23346 heartbeater.cc:646] Failed to heartbeat to 127.21.150.254:44861 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.150.254:44861 timed out after 15.000s (SENT)
I20260709 12:15:17.667814 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.646646 24016 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:17.646966 28487 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:17.669212 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45602.3us
I20260709 12:15:17.652997 24014 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 7.
I20260709 12:15:17.662891 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2390) took 2557 ms. Trace:
I20260709 12:15:17.664793 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.665936 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.670486 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:17.673399 24011 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:17.673655 28375 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:17.675259 23468 rpcz_store.cc:276] 0709 12:15:15.105598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.003014 (+897416us) service_pool.cc:224] Handling call
0709 12:15:16.021046 (+ 18032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:16.156664 (+135618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.156695 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.483712 (+1327017us) spinlock_profiling.cc:243] Waited 6.92 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:17.503901 (+ 20189us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.569103 (+ 65202us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:17.569144 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.569216 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:17.640035 (+ 70819us) raft_consensus.cc:1685] finished
0709 12:15:17.640057 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:17.662816 (+ 22759us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110889,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":36359,"spinlock_wait_cycles":15906176,"thread_start_us":136531,"threads_started":2,"wal-append.queue_time_us":115327}
I20260709 12:15:17.679103 28385 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.680629 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.682915 28477 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.678164 28359 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:17.681841 23468 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:17.685123 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.685801 23468 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.686405 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:17.687005 23464 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.694396 28412 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:17.694281 28487 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.694495 28477 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:17.695636 28477 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:17.697152 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2459312990001763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 12:15:17.697639 23287 connection.cc:256] RPC call timeout handler was delayed by -3.2423312990001705s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 12:15:17.697676 28503 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:17.698666 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0812312990001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
W20260709 12:15:17.699074 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0668312990001141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 12:15:17.698946 28503 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:17.700387 28477 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.700695 28508 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:17.701906 28508 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.702499 23468 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:17.695433 28412 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:17.695433 28487 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.705984 28339 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:17.707257 28487 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.704464 23468 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:17.674445 28407 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.715651 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.716145 28113 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:17.716398 23468 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:17.718554 28385 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.719143 23468 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:17.703778 23467 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 12:15:17.699852 23287 connection.cc:256] RPC call timeout handler was delayed by -3.0065312990000166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
I20260709 12:15:17.713155 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:17.722043 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9345312990001275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
W20260709 12:15:17.722445 23287 connection.cc:256] RPC call timeout handler was delayed by -2.9318312990001232s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 12:15:17.722772 23287 connection.cc:256] RPC call timeout handler was delayed by -2.782431299000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 12:15:17.723470 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7752312990000974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
W20260709 12:15:17.724108 23287 connection.cc:256] RPC call timeout handler was delayed by -2.7482312990000537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
I20260709 12:15:17.723600 23468 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:15:17.724629 23287 connection.cc:256] RPC call timeout handler was delayed by -2.5520312990001912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22719999999999999s for a response
W20260709 12:15:17.724985 23287 connection.cc:256] RPC call timeout handler was delayed by -1.9742312990001665s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29140000000000005s for a response
W20260709 12:15:17.725415 23287 connection.cc:256] RPC call timeout handler was delayed by -1.943631299000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2948s for a response
I20260709 12:15:17.726218 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.737118 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:17.740871 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:17.742767 28463 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:17.746388 23315 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:17.744545 28049 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:17.746984 28049 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:17.747371 28049 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.744998 28351 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:17.743933 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:17.756177 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.734393 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:17.777318 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.779099 23312 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
W20260709 12:15:17.788231 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:17.807039 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:17.807794 23315 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:17.809515 23464 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:17.816187 23312 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:15:17.816586 23312 pending_rounds.cc:85] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:15:17.816793 23312 pending_rounds.cc:107] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:15:17.817018 23312 raft_consensus.cc:2889] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:17.716745 28113 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.813687 28359 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:17.826819 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49600.3us
I20260709 12:15:17.828404 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.813618 28316 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:17.817832 28311 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.793443 28404 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:17.838291 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.834632 28465 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.840265 28465 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:17.831980 28477 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:17.830018 28359 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:17.842963 23461 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:17.843186 28359 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:17.843536 28402 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:17.844056 28402 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.849630 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.850689 28311 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:17.852145 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.853607 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.854295 23315 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:17.854642 28311 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:17.855922 28311 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.857847 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.859555 28311 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:17.861711 28465 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.863447 23214 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 9, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 9 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:17.868683 28455 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.870069 28508 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.871013 23313 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:15:17.867048 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.873914 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.869158 28113 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:17.890959 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.834971 28404 consensus_queue.cc:1243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:17.915652 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:17.916476 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 180025us
I20260709 12:15:17.925069 23319 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:17.925266 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.891952 28454 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.904314 28455 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:17.931274 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 180061us
I20260709 12:15:17.889928 28049 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:17.941887 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.948761 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:17.928874 28465 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:17.952217 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:17.952795 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.953473 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.952026 23387 raft_consensus.cc:2410] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:17.949405 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.956535 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:17.956773 28311 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:17.956907 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:17.957998 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1031) took 2447 ms. Trace:
I20260709 12:15:17.959800 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:17.958477 23466 rpcz_store.cc:276] 0709 12:15:15.510492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.414105 (+903613us) service_pool.cc:224] Handling call
0709 12:15:16.414482 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:16.574935 (+160453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:16.574983 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:16.575040 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:16.575100 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.957777 (+1382677us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002a03d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:17.957822 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:17.957935 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368423296}
I20260709 12:15:17.963855 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.970389 23389 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:17.969280 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:18.003067 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.005118 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.008108 23466 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:15:18.021790 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:18.025638 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68798us
I20260709 12:15:18.025755 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.026791 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:17.952307 28454 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:18.033519 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.037012 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:17.965498 28407 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:17.965937 28463 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.001016 28316 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:18.047032 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.144400 23314 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:18.255115 23360 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.098s (SENT) [suppressed 32 similar messages]
I20260709 12:15:18.257337 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:18.280081 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 553916us
I20260709 12:15:18.027912 28303 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:17.960691 28339 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
W20260709 12:15:18.297937 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 250986us
I20260709 12:15:18.305262 23316 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:18.031291 28311 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:18.316953 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.325863 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.042335 28302 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:18.324093 23314 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:18.338124 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 311416us
W20260709 12:15:18.342657 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.343117 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.348361 23315 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:18.348789 23318 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:18.349208 23318 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:18.139374 28450 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.350759 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.031651 28454 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:18.362706 23314 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:18.363353 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:18.375603 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58782.7us
I20260709 12:15:18.030530 28465 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:18.425941 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62675.1us
I20260709 12:15:18.312116 28391 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:18.280776 28473 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:18.375104 23314 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:18.034214 28298 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:18.357167 23318 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:18.323717 28407 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:18.044625 28042 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.409s
I20260709 12:15:18.324856 28311 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:18.375559 28514 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.503s	user 0.000s	sys 0.001s
I20260709 12:15:18.357007 28508 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:18.363071 28495 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:18.468644 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.424736 28375 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.425902 28454 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:18.504962 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.506563 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.481964 28440 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.487650 23318 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:18.482129 28450 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:18.359273 28463 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.124s
W20260709 12:15:18.501631 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48967.2us
I20260709 12:15:18.426141 28455 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:18.487861 28311 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:18.542042 28527 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:15:18.541860 28432 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:18.543310 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2420) took 1346 ms. Trace:
I20260709 12:15:18.543676 23463 rpcz_store.cc:276] 0709 12:15:17.196891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.355926 (+159035us) service_pool.cc:224] Handling call
0709 12:15:17.356504 (+   578us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.356939 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.356985 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.752497 (+395512us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b18002fd280. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a
0709 12:15:17.753515 (+  1018us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:17.754047 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.954942 (+200895us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:17.954977 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.955041 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:18.543085 (+588044us) raft_consensus.cc:1685] finished
0709 12:15:18.543104 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:18.543254 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2236,"raft.queue_time_us":87,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4474,"spinlock_wait_cycles":27641088,"thread_start_us":200510,"threads_started":1,"wal-append.queue_time_us":287731}
I20260709 12:15:18.545571 28049 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:18.546061 28339 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:18.546617 28339 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.548671 28339 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:18.549762 28518 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:18.522822 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.482712 28113 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:18.550035 28520 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.525s	user 0.001s	sys 0.000s
I20260709 12:15:18.521915 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.482945 28391 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.491832 28473 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:18.591252 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 69454.4us
I20260709 12:15:18.549763 28342 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:15:18.557351 28516 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.596s	user 0.001s	sys 0.001s
I20260709 12:15:18.604552 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.604668 23318 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:15:18.550414 28518 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.519636 28440 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.551942 28382 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 12:15:18.618302 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:18.622285 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 161809us
I20260709 12:15:18.531229 28450 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.586112 28508 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:18.646124 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53340.1us
I20260709 12:15:18.644886 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:18.598922 28342 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:18.652798 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 204214us
I20260709 12:15:18.658420 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.672873 28049 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:18.677264 28049 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:18.678700 28339 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:18.679145 28339 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:18.680116 28339 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.693049 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:18.693997 23466 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:18.694698 28518 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:18.695410 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:18.700336 28503 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:18.615033 28386 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:18.576263 28113 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:18.704672 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57530us
I20260709 12:15:18.705955 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:18.728828 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.730676 23320 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:15:18.707242 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.731141 23320 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:18.733629 23314 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:18.734613 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:18.724797 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 197168us
I20260709 12:15:18.736496 23320 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:18.742789 23320 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:18.734648 28535 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:15:18.717759 28386 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:18.760257 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 417244us
I20260709 12:15:18.748616 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:18.739861 28503 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:18.761325 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.736558 28523 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:18.741780 28522 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:18.889670 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:18.890749 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:18.906523 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:18.748467 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.002s	sys 0.019s
W20260709 12:15:18.925554 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.003s	sys 0.019s
I20260709 12:15:18.930895 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.932698 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:18.934554 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:18.984088 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:18.985064 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:18.900518 28522 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:18.993145 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:18.998979 24008 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:18.996335 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:18.752941 28342 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:18.956158 28316 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:18.959859 28455 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:18.956812 28042 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:19.015341 28539 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:15:19.013968 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 79521.5us
I20260709 12:15:18.956326 28543 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:19.023912 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.025830 28503 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:19.026595 28503 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.027591 23320 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:15:19.034197 28542 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:15:19.011413 28522 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.073832 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.076399 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
W20260709 12:15:19.075964 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52164.8us
I20260709 12:15:19.080153 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.021775 28518 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:19.061800 28538 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 12:15:19.018594 28473 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.112166 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.024920 28342 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.121908 24007 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:19.029306 28503 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.119482 24009 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:19.075135 28440 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.107492 28455 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.132405 28455 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:19.132931 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2575) took 1301 ms. Trace:
I20260709 12:15:19.133242 28455 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:19.136176 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2589) took 1188 ms. Trace:
I20260709 12:15:19.137799 28391 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.142163 28386 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.133381 23313 rpcz_store.cc:276] 0709 12:15:17.831082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.831273 (+   191us) service_pool.cc:224] Handling call
0709 12:15:17.870789 (+ 39516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.871379 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.132723 (+1261344us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b540001d950. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:19.132875 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90116480}
I20260709 12:15:19.146893 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.136731 23316 rpcz_store.cc:276] 0709 12:15:17.947755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.335841 (+388086us) service_pool.cc:224] Handling call
0709 12:15:18.348226 (+ 12385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.364266 (+ 16040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.364311 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.439650 (+ 75339us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.734988 (+295338us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:18.735026 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:18.735118 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:19.135898 (+400780us) raft_consensus.cc:1685] finished
0709 12:15:19.135925 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.136102 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79602,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":16566,"thread_start_us":240186,"threads_started":1,"wal-append.queue_time_us":515341}
W20260709 12:15:19.150089 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.150995 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.114076 23317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.016s	sys 0.008s
W20260709 12:15:19.157850 23317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.016s	sys 0.008s
I20260709 12:15:19.162413 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2578) took 1257 ms. Trace:
I20260709 12:15:19.163414 23317 rpcz_store.cc:276] 0709 12:15:17.905222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.905696 (+   474us) service_pool.cc:224] Handling call
0709 12:15:17.926694 (+ 20998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.926899 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:18.352415 (+425516us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.161929 (+809514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.162009 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:19.162072 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.162165 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.162334 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4149,"prepare.queue_time_us":581734,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":1212342,"threads_started":2}
I20260709 12:15:19.123769 28042 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.173903 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:19.288177 23316 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 15 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:19.124102 28113 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.111248 28450 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.128182 28503 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.292150 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:19.295332 23319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.016s	sys 0.032s
W20260709 12:15:19.299319 23319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.017s	sys 0.033s
W20260709 12:15:19.289090 24016 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.019s	sys 0.030s
I20260709 12:15:19.301011 23313 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:19.300383 24016 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.019s	sys 0.030s
I20260709 12:15:19.121057 28518 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:19.305064 28548 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:15:19.295248 28477 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:19.316861 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.047s
W20260709 12:15:19.317391 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.008s	sys 0.047s
W20260709 12:15:19.318115 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 582124us
I20260709 12:15:19.292799 28042 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:19.325443 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.334937 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 586375us
I20260709 12:15:19.366976 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:19.373509 23461 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2410) took 2471 ms (client timeout 1573 ms). Trace:
W20260709 12:15:19.374073 23461 rpcz_store.cc:269] 0709 12:15:16.897680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:16.897790 (+   110us) service_pool.cc:224] Handling call
0709 12:15:19.368966 (+2471176us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b5400226fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:19.369123 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169561984}
I20260709 12:15:19.375144 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:19.378536 23319 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:15:19.379120 23319 pending_rounds.cc:85] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:19.379447 23319 pending_rounds.cc:107] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:15:19.379798 23319 raft_consensus.cc:2889] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:19.384765 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:19.386693 28556 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:15:19.387720 28042 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.384033 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:19.387668 28373 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:19.392261 28373 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.392527 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:19.392675 28373 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:19.393162 24009 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.361150 28440 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:15:19.394186 23319 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:19.376461 28454 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.002s
I20260709 12:15:19.410399 28440 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:19.410694 28454 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.413471 28454 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:19.389052 28549 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:19.416414 28311 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.846s	user 0.013s	sys 0.000s
I20260709 12:15:19.416258 28549 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.398495 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:19.380369 28495 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.876s	user 0.008s	sys 0.004s
I20260709 12:15:19.375530 28385 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:19.318784 28477 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.422783 28375 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:19.423772 28375 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.426143 28117 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:19.425946 28385 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:19.445951 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.440198 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.416992 28311 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.412220 28473 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:15:19.434787 28359 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.589s	user 0.015s	sys 0.002s
I20260709 12:15:19.426640 28495 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:19.387892 28546 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
W20260709 12:15:19.456530 23362 connection.cc:256] RPC call timeout handler was delayed by -1.2133536369999547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:15:19.443222 23312 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.007s	sys 0.033s
W20260709 12:15:19.459981 23312 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.007s	sys 0.033s
W20260709 12:15:19.443348 28550 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
W20260709 12:15:19.453454 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 373380us
I20260709 12:15:19.477681 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2593) took 1429 ms. Trace:
I20260709 12:15:19.478564 23315 rpcz_store.cc:276] 0709 12:15:18.048480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.349526 (+301046us) service_pool.cc:224] Handling call
0709 12:15:18.373976 (+ 24450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:18.374299 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:15:18.610630 (+236331us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.477079 (+866449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.477163 (+    84us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:19.477239 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.477327 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.477622 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53072,"prepare.queue_time_us":161298,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":70062,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"thread_start_us":1008861,"threads_started":3,"wal-append.queue_time_us":695483}
I20260709 12:15:19.474043 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:19.482338 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:19.461665 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2574) took 1637 ms. Trace:
I20260709 12:15:19.476194 24009 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:19.476366 24013 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:19.484051 24012 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:19.477986 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.477820 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.483227 23312 rpcz_store.cc:276] 0709 12:15:17.824278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.826137 (+  1859us) service_pool.cc:224] Handling call
0709 12:15:17.826411 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.826597 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:15:17.950878 (+124281us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.460702 (+1509824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.460779 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:19.460854 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.461335 (+   481us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.461543 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66225,"prepare.queue_time_us":485739,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":210735,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":1622377,"threads_started":3,"wal-append.queue_time_us":1056994}
I20260709 12:15:19.472886 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:19.475031 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:19.492975 23395 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:19.474043 23387 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:19.482630 23389 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:19.451251 28385 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.446120 28454 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:19.534111 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56198.8us
I20260709 12:15:19.538726 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:19.551903 23389 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:19.551915 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.444415 28117 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:19.553532 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.553663 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:19.557739 23982 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.009s	sys 0.017s
I20260709 12:15:19.557710 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:19.558355 23982 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.009s	sys 0.017s
I20260709 12:15:19.560074 24009 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:19.562438 23287 leader_election.cc:400] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:19.562877 23287 leader_election.cc:403] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:19.564006 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.564103 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.564718 23463 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
W20260709 12:15:19.573791 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 406252us
W20260709 12:15:19.575824 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.576834 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.581633 28557 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.582312 28557 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.584885 28303 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.585026 28557 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.585624 28113 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.593940 28402 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.594553 28402 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.459546 28387 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:19.594441 28113 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.586946 28503 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.597802 28503 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.600584 28503 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.594002 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:19.603380 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:19.595268 28387 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:19.604605 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.604856 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:19.605748 28351 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:19.606140 28351 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:19.606731 28387 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.585419 28303 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.607100 28351 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.608198 28351 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:19.608976 28303 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:19.588110 28508 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:19.595242 28402 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.611724 28402 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:19.611945 28117 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.612960 28117 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:19.551098 28450 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:19.625821 28450 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:19.626354 28450 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:19.634995 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:19.575856 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:19.587826 28526 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.853s	user 0.001s	sys 0.019s
I20260709 12:15:19.640439 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.642138 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:19.574488 23286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.016s	sys 0.010s
I20260709 12:15:19.641330 23466 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:19.601267 28503 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:19.458050 28359 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.657189 28375 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.657896 28375 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:19.658391 28375 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:19.658593 28311 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.659389 28375 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.661600 24012 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:19.650611 28565 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.663414 28565 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.643002 28560 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.665877 28560 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.667999 28560 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.669080 28477 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:19.653682 28553 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:15:19.671813 23469 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:19.565282 28455 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:19.567457 28042 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:19.651895 28552 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 12:15:19.652050 28557 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:19.682358 24014 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 14. Ops: []
I20260709 12:15:19.561347 28454 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:19.674855 28455 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.699107 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1103) took 1662 ms. Trace:
I20260709 12:15:19.702359 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2436) took 1896 ms. Trace:
W20260709 12:15:19.703662 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.643409 23286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.016s	sys 0.011s
I20260709 12:15:19.702838 23467 rpcz_store.cc:276] 0709 12:15:17.806555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.806728 (+   173us) service_pool.cc:224] Handling call
0709 12:15:17.807068 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:17.819665 (+ 12597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:17.819696 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:17.828086 (+  8390us) log.cc:844] Serialized 26 byte log entry
0709 12:15:17.955246 (+127160us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:17.955287 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:17.955365 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:19.587155 (+1631790us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:19.700186 (+113031us) raft_consensus.cc:1685] finished
0709 12:15:19.700221 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.702293 (+  2072us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6910,"raft.queue_time_us":8310,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4380,"spinlock_wait_cycles":6369664,"thread_start_us":93501,"threads_started":1,"wal-append.queue_time_us":823548}
I20260709 12:15:19.643502 23470 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:19.642975 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2431) took 1985 ms. Trace:
I20260709 12:15:19.708583 23468 rpcz_store.cc:276] 0709 12:15:17.657739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.720090 (+ 62351us) service_pool.cc:224] Handling call
0709 12:15:17.722821 (+  2731us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.724802 (+  1981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.642764 (+1917962us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b54002251d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:19.642906 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5432064}
W20260709 12:15:19.709173 23320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.001s	sys 0.032s
W20260709 12:15:19.710546 23320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.002s	sys 0.032s
I20260709 12:15:19.699740 28557 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:19.643616 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 259840us
W20260709 12:15:19.701552 28342 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.003s
I20260709 12:15:19.677809 28503 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:19.699438 28084 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:19.695461 28565 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.702253 24016 rpcz_store.cc:276] 0709 12:15:18.036496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.038003 (+  1507us) service_pool.cc:224] Handling call
0709 12:15:18.282706 (+244703us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:18.298618 (+ 15912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:18.298648 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:18.323547 (+ 24899us) log.cc:844] Serialized 26 byte log entry
0709 12:15:18.636612 (+313065us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:19.302111 (+665499us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.697706 (+395595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.698716 (+  1010us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:19.698785 (+    69us) raft_consensus.cc:1685] finished
0709 12:15:19.698807 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.699025 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16254,"prepare.queue_time_us":777227,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":170177,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":19412,"thread_start_us":1212058,"threads_started":3,"wal-append.queue_time_us":407995}
I20260709 12:15:19.698500 28455 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:19.643708 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64616.7us
I20260709 12:15:19.699201 28113 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:19.729686 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 271401us
I20260709 12:15:19.643744 28518 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:19.647207 28356 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.722087 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.712466 28477 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:19.733325 28503 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:19.735352 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:19.714702 28560 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:19.732507 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:19.739362 28531 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.204s	user 0.002s	sys 0.025s
I20260709 12:15:19.747049 28454 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:19.748332 28572 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:19.750114 23315 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:19.750537 23312 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:19.732478 28568 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:19.731164 28386 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:15:19.748428 28356 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:19.755831 28522 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 12:15:19.765200 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.765550 23469 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:19.761785 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 157003us
I20260709 12:15:19.766834 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:19.764429 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.763038 23319 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:19.767887 23468 raft_consensus.cc:2410] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:19.732286 28573 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:15:19.768496 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.768636 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:19.756455 28118 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:19.770612 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.770968 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:19.771247 23469 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:19.771569 23467 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:19.772295 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.776135 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:19.753332 28477 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.754097 28560 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.788496 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:19.765607 28454 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:19.757503 28572 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.791965 28454 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.787567 28564 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:19.794138 28339 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:19.795960 28560 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.796722 28560 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:19.800566 28454 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.785220 28557 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.775619 28376 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:15:19.823558 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1102) took 1787 ms. Trace:
W20260709 12:15:19.824755 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 52549us
I20260709 12:15:19.811193 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1316) took 2071 ms. Trace:
I20260709 12:15:19.786449 28113 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:15:19.826373 24014 rpcz_store.cc:276] 0709 12:15:18.035953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.037914 (+  1961us) service_pool.cc:224] Handling call
0709 12:15:18.254739 (+216825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.683967 (+1429228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.823273 (+139306us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400775e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:19.823473 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282841984}
I20260709 12:15:19.827688 23321 rpcz_store.cc:276] 0709 12:15:17.739889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:17.740060 (+   171us) service_pool.cc:224] Handling call
0709 12:15:17.743500 (+  3440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:17.836324 (+ 92824us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:18.000174 (+163850us) log.cc:844] Serialized 14 byte log entry
0709 12:15:18.447604 (+447430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:18.447672 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:18.447743 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.136773 (+689030us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18003f71e0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:19.810859 (+674086us) spinlock_profiling.cc:243] Waited 92.6 ms on lock 0x7b540001cf54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:19.810935 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:19.811116 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60771,"prepare.queue_time_us":541522,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":37191,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":13368,"spinlock_wait_cycles":220819584,"thread_start_us":588742,"threads_started":2,"wal-append.queue_time_us":761949}
I20260709 12:15:19.826785 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:19.790846 28508 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:19.797271 28560 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:19.770103 28522 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.793831 28572 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.843921 28564 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:19.807184 28546 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:19.819422 28359 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.767712 28568 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.853188 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.790732 28450 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.806490 28574 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:15:19.840111 28113 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:19.859084 23318 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:19.871912 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 100852us
I20260709 12:15:19.839191 28579 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.832427 28385 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:19.858357 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 90512.5us
W20260709 12:15:19.878429 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 90185us
W20260709 12:15:19.881042 23314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2599) took 1439 ms (client timeout 1881 ms). Trace:
W20260709 12:15:19.881500 23314 rpcz_store.cc:269] 0709 12:15:18.444794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.522576 (+ 77782us) service_pool.cc:224] Handling call
0709 12:15:19.880872 (+1358296us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400388e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:19.880984 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385123072}
I20260709 12:15:19.856647 28386 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:19.883786 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:19.884635 28578 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:19.884969 28560 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.886119 28450 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.830260 28503 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.887653 28549 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.858621 28546 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.862414 28477 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.890265 28546 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.899467 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:19.903255 23461 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:15:19.904685 23461 raft_consensus.cc:606] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:19.904889 28588 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:19.905122 23461 raft_consensus.cc:614] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:19.906032 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.904685 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:19.904645 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:19.891321 28386 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:19.972049 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 67478.2us
W20260709 12:15:19.978130 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:19.975392 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 70841.4us
I20260709 12:15:19.977052 28568 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:19.906733 28579 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:19.968609 28113 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:19.973877 28382 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:19.999598 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:19.979817 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:19.973996 28412 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:19.977746 28386 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:19.978750 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:20.012344 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:19.986568 28522 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.014446 28585 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:20.014946 28585 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:20.015388 28585 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:20.015722 28373 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
I20260709 12:15:20.016175 28373 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.017274 24008 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:20.012084 28568 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:20.019493 28584 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:20.021644 28356 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.001413 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.022997 24009 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:15:20.024300 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.026423 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:19.982936 28588 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:20.027070 23389 raft_consensus.cc:606] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:20.028458 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.000115 28382 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.032287 23387 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:20.023487 28440 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:20.032650 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.034102 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.034410 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.035827 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:20.036816 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 158401us
I20260709 12:15:20.029573 28316 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.030076 28568 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:20.048985 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.054746 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:19.991073 28564 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.056056 28373 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:20.038005 28382 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.037873 28440 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.037662 28503 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:20.064045 28508 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:20.060748 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.041985 28455 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.060702 28560 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:15:20.086280 28566 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.326s	user 0.003s	sys 0.000s
I20260709 12:15:20.112262 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2601) took 1635 ms. Trace:
I20260709 12:15:20.112519 28455 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:20.112692 23320 rpcz_store.cc:276] 0709 12:15:18.477132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.028528 (+551396us) service_pool.cc:224] Handling call
0709 12:15:19.028912 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:19.029280 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:19.118627 (+ 89347us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.713005 (+594378us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:19.719120 (+  6115us) log.cc:844] Serialized 26 byte log entry
0709 12:15:19.719308 (+   188us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:19.789577 (+ 70269us) log.cc:844] Serialized 14 byte log entry
0709 12:15:19.789800 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.789936 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:20.085610 (+295674us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:20.112015 (+ 26405us) raft_consensus.cc:1685] finished
0709 12:15:20.112040 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.112197 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":651,"prepare.queue_time_us":221194,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":114687,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"spinlock_wait_cycles":3472256,"thread_start_us":616509,"threads_started":2,"wal-append.queue_time_us":606091}
W20260709 12:15:20.114898 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54201.4us
I20260709 12:15:20.116106 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.117859 28568 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:20.118360 28568 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.118813 28568 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:20.123083 28455 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.123718 28455 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.125579 28560 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:20.127424 28455 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.128403 28560 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.129607 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.053367 28316 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.084081 28440 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.133512 23388 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:20.133973 28042 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:20.134184 28590 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:15:20.136924 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 83956.9us
I20260709 12:15:20.131430 28560 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.066252 28557 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:20.139122 23388 raft_consensus.cc:606] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:20.141891 28042 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.142700 28584 raft_consensus.cc:993] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:15:20.149973 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 96189us
I20260709 12:15:20.158202 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:20.152626 28585 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.154467 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.155941 23388 raft_consensus.cc:614] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:20.171937 28585 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.173688 28118 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:20.176236 28118 raft_consensus.cc:1081] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:20.154925 28522 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:15:20.184288 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.178669 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.186378 23391 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:20.190603 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.194916 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.200551 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.201583 23470 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
W20260709 12:15:20.202354 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.203625 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.164750 28358 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:20.205884 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.209378 23470 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:20.209616 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.210592 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.211378 23464 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:20.211278 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.212596 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:20.213882 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.216817 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1053) took 1467 ms. Trace:
I20260709 12:15:20.219053 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.220243 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:20.221964 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:20.223112 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.223305 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.221172 23314 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:20.226699 23461 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:20.183203 28455 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.229462 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:20.229712 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.197786 28578 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:20.225024 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.182015 28557 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:20.239784 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.226809 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.207926 28454 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.224745 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.233651 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.234602 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.244942 23317 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:15:20.244801 23315 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:20.246044 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.247552 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.199196 28503 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:20.238863 23320 pending_rounds.cc:85] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:20.239629 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.252028 23320 pending_rounds.cc:107] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:15:20.219245 23465 rpcz_store.cc:276] 0709 12:15:18.749811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:18.749983 (+   172us) service_pool.cc:224] Handling call
0709 12:15:19.020321 (+270338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.656999 (+636678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.657034 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.657075 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:19.657106 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:20.216602 (+559496us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54004056d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:20.216648 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.216764 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463692160}
I20260709 12:15:20.252315 23320 raft_consensus.cc:2889] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:20.252691 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:20.240937 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.254463 23389 raft_consensus.cc:606] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:20.254559 23387 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.239629 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.255291 23321 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:20.256186 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.240454 23313 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:20.212556 28358 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.250923 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:20.252215 23388 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:20.225828 28564 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.210556 28450 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:15:20.227703 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:20.258077 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.253837 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:20.249441 28595 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:15:20.264011 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.263970 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.266891 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.268004 23318 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:20.266563 23317 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:20.268498 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:20.259369 28358 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.239632 28455 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.264745 28578 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.296492 28578 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.250597 28454 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.301257 28454 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.277715 28522 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:20.305779 28522 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.305171 28560 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.306190 28522 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:20.306702 28455 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:20.289271 28382 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.291168 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:20.293228 28554 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.880s	user 0.006s	sys 0.000s
I20260709 12:15:20.309132 28560 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.277456 28508 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:20.312474 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:20.313665 23314 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:20.263177 28564 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.316015 23394 raft_consensus.cc:606] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:20.271147 28557 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:20.317389 28508 consensus_queue.cc:1243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:20.317807 28592 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:20.283888 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:20.325160 23390 raft_consensus.cc:2368] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14.
I20260709 12:15:20.323055 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:20.345491 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 61643.4us
I20260709 12:15:20.347874 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.351150 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.319087 28579 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:20.353204 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.355068 23319 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:20.357349 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.358093 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.360016 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.322350 28557 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.314985 28568 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.371338 23319 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
W20260709 12:15:20.423612 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.354828 28576 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:20.353847 28455 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:20.439466 28576 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:20.436744 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 65964.3us
I20260709 12:15:20.380456 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:20.412135 23980 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:20.356923 28316 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.365168 28450 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:20.503521 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:20.495111 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.482374 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.489362 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.500274 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.494879 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.471132 23214 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:20.363920 28454 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:20.538245 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 126231us
I20260709 12:15:20.493489 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.366317 28382 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.550856 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:20.364122 28557 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.581446 23464 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:20.576808 24010 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:20.516594 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.583070 24011 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:15:20.532914 23465 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:15:20.465754 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:20.451198 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.547144 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.502743 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1322) took 1348 ms. Trace:
W20260709 12:15:20.584744 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 219907us
I20260709 12:15:20.442054 23394 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:20.577926 24012 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:20.577926 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:20.585209 23316 rpcz_store.cc:276] 0709 12:15:19.154821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:19.290250 (+135429us) service_pool.cc:224] Handling call
0709 12:15:19.882454 (+592204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:19.882655 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:19.882672 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:19.882700 (+    28us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:19.882719 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:19.882755 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:20.502545 (+619790us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400388e50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:20.502678 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354913152}
I20260709 12:15:20.586444 23394 raft_consensus.cc:614] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:20.588982 23387 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:20.589666 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.373585 28560 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:20.575747 24008 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 7: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:20.580137 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:20.595695 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.596287 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.596886 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:20.591228 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.598635 24010 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:20.494320 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.381035 28579 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:20.596946 24012 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:20.422585 28508 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.378703 28042 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:20.475569 28331 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.596391 24007 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:20.607146 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.608153 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:20.602695 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:20.603533 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.380916 28585 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:20.516449 28455 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:20.423522 28463 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.611177 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.532840 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.405833 28578 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.513604 28576 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.576202 28541 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:20.611898 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.626097 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:20.626710 24007 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.614452 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:20.471340 28564 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.465751 28546 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.149s
I20260709 12:15:20.503028 28359 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.503980 28351 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.615927 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:20.634985 23470 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.585708 28454 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.574486 28450 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:20.635864 23218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 318) took 16532 ms (client timeout 15000 ms). Trace:
I20260709 12:15:20.636029 28450 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:20.636328 23218 rpcz_store.cc:269] 0709 12:15:03.924499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:03.929059 (+  4560us) service_pool.cc:224] Handling call
0709 12:15:04.218794 (+289735us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:20.440690 (+16221896us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:10.070615 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:10.210160 (+139545us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:10.219533 (+  9373us) write_op.cc:435] Acquired schema lock
0709 12:15:10.221607 (+  2074us) tablet.cc:662] Decoding operations
0709 12:15:10.870105 (+648498us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:10.870227 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:10.870271 (+    44us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:15:10.872609 (+  2338us) tablet.cc:701] Row locks acquired
0709 12:15:10.872635 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:15:10.872854 (+   219us) write_op.cc:270] Start()
0709 12:15:10.873026 (+   172us) write_op.cc:276] Timestamp: P: 1783599310872781 usec, L: 0
0709 12:15:10.873058 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:15:11.104170 (+231112us) log.cc:844] Serialized 8706 byte log entry
0709 12:15:11.655891 (+551721us) op_driver.cc:464] REPLICATION: finished
0709 12:15:11.740394 (+ 84503us) write_op.cc:301] APPLY: starting
0709 12:15:11.740481 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:11.752771 (+ 12290us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:11.752918 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:11.861984 (+109066us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:11.862599 (+   615us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 12:15:11.862696 (+    97us) write_op.cc:312] APPLY: finished
0709 12:15:11.863600 (+   904us) log.cc:844] Serialized 196 byte log entry
0709 12:15:11.865793 (+  2193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:11.866545 (+   752us) write_op.cc:454] Released schema lock
0709 12:15:11.866626 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:11.882627 (+ 16001us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:11.882668 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1157066,"reported_tablets":47,"spinlock_wait_cycles":4874624,"child_traces":[["op",{"apply.queue_time_us":76180,"apply.run_cpu_time_us":33637,"apply.run_wall_time_us":142078,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96175,"num_ops":22,"prepare.queue_time_us":496290,"prepare.run_cpu_time_us":80032,"prepare.run_wall_time_us":1304688,"raft.queue_time_us":167932,"raft.run_cpu_time_us":18419,"raft.run_wall_time_us":154360,"replication_time_us":733813,"thread_start_us":725140,"threads_started":4,"wal-append.queue_time_us":249322,"wal-append.run_cpu_time_us":30958,"wal-append.run_wall_time_us":1517690}]]}
I20260709 12:15:20.608222 28356 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.621115 23214 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 9, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:20.472141 28440 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.465576 28437 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.148s
I20260709 12:15:20.594676 28503 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:20.602984 28391 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:20.623575 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:20.627686 28541 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:20.624116 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.442934 28432 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.175s
I20260709 12:15:20.577734 28316 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:20.669804 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45798.9us
I20260709 12:15:20.672161 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
W20260709 12:15:20.661545 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71965.6us
I20260709 12:15:20.674597 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.661966 23214 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 11, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 11 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:20.677879 23470 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:20.678619 23214 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:20.622433 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:20.613284 28455 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.658617 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:20.534042 28386 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.654281 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:20.654809 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 294837us
I20260709 12:15:20.630719 28578 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:20.691165 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.602321 28518 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:15:20.604857 28560 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.635883 28454 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.610832 28331 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.618960 28508 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.656682 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:20.682523 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.626787 28382 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:15:20.623170 28596 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:15:20.691448 28455 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:20.654973 28356 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.626514 28463 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.685812 28579 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.705561 28564 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:20.707023 28522 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.664367 28555 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:15:20.663405 28541 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:20.712826 28608 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:15:20.656931 28391 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.681191 28528 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:20.703266 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:20.655524 28351 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.681675 28568 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.656270 28503 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:20.681932 28473 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:15:20.726233 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 69614.3us
I20260709 12:15:20.791071 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.703962 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:20.715399 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 114627us
I20260709 12:15:20.792610 23470 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 6.
I20260709 12:15:20.704710 28578 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.743553 28564 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:20.707453 28560 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.742841 24014 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:20.653702 28359 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.686337 28316 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.752122 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.752149 24013 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:20.740181 28562 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:20.707206 28486 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:15:20.789299 24010 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:20.812702 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.813597 23312 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:20.742421 28609 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:20.709574 28588 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:20.791900 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:20.722755 28373 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:15:20.825305 23313 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:20.826040 28373 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:20.826432 28373 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:20.827348 28373 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.828646 28373 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.829393 28555 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:20.829891 28356 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.829993 28555 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:20.831521 28358 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.001s
I20260709 12:15:20.831090 28555 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.831988 28358 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.833814 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.835144 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.840063 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.842303 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:20.796185 28473 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:20.780534 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:20.756506 28612 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.852084 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:20.852619 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.855353 23313 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:20.856349 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.805006 23465 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:15:20.805575 23316 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:15:20.856851 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 87332.2us
I20260709 12:15:20.818017 24009 raft_consensus.cc:2410] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:15:20.799467 28503 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:20.799248 28578 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:20.800194 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 117779us
I20260709 12:15:20.803239 28564 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.849308 23470 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:20.758179 28391 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:20.865835 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:20.801868 28568 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.872557 23312 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:20.884856 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.885421 23469 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:20.805006 28562 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:20.843991 28611 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:20.879703 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:20.881119 23392 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:20.882187 23387 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:20.877621 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:20.835776 28588 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:20.891775 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:20.888119 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 184911us
I20260709 12:15:20.894435 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.890401 23392 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:20.902720 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.908759 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:20.910878 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 117071us
I20260709 12:15:20.912197 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:20.913784 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.913784 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:20.869987 28356 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.919801 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:20.882396 28385 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:20.932049 23312 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:20.926365 23392 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.960384 23392 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:20.890359 28562 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:20.919067 28546 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:20.935230 28574 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:20.935915 28440 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.890558 28568 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:20.845896 28609 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:21.005988 28617 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:15:21.006448 28609 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.928512 28573 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:15:20.981742 28113 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:20.875764 28612 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:20.963022 28356 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.026834 23314 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:20.996305 28358 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:20.978596 28450 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.035043 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:21.038332 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.009490 28574 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:21.045368 23469 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:20.990994 28632 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:21.047699 28632 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:21.048126 28632 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.059329 23314 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:21.032557 28042 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:21.031451 28522 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.061949 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:21.030889 28463 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.034562 28331 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.041359 28634 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:20.988647 28473 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.065320 24007 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.068073 23466 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:21.030035 28113 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.097397 24007 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:21.069550 28316 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.071115 28634 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.101069 28634 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:21.109628 28359 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.114152 28473 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.115186 28568 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.116078 28568 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:21.127929 23392 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:21.075055 28042 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:21.060613 28541 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.141390 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 904) took 1107 ms. Trace:
I20260709 12:15:21.146286 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:21.147104 23319 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:21.149855 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 115118us
I20260709 12:15:21.151219 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:21.141999 23391 rpcz_store.cc:276] 0709 12:15:20.033597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.033841 (+   244us) service_pool.cc:224] Handling call
0709 12:15:21.141116 (+1107275us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540005ad54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:21.141310 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318455552}
I20260709 12:15:21.141778 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:21.156018 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:21.185875 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44195us
I20260709 12:15:21.187700 24007 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:21.188381 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:21.134889 28376 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 12:15:21.191735 28578 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.192314 28391 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.181118 28636 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:21.193629 24012 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:21.193885 28626 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.194360 28636 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:21.194449 28626 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.195535 28331 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:15:21.195815 28625 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:15:21.196035 28331 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:21.196496 28331 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:21.196925 28626 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.155083 28624 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.199011 28624 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.199757 28541 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.200973 28612 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:21.201133 28624 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:21.202412 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46536.4us
I20260709 12:15:21.229338 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:21.139768 28621 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:15:21.213413 23319 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:21.232425 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:21.155804 28622 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:15:21.227938 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:21.264933 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:21.272154 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:21.276688 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:21.279942 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:21.282836 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:21.131247 28564 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.301110 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:21.304245 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:21.305207 23468 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:15:21.394487 23214 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.000s	sys 0.015s
W20260709 12:15:21.395152 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 500975us
I20260709 12:15:21.396822 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:21.397931 23394 raft_consensus.cc:2410] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:15:21.397847 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 484126us
I20260709 12:15:21.402678 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:21.394908 23214 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.000s	sys 0.015s
I20260709 12:15:21.284871 28636 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:21.278236 28624 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:21.281376 28440 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.285075 28568 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:21.421912 23317 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:21.426134 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:21.428282 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:21.430714 23313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.014s
W20260709 12:15:21.431908 23313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.006s	sys 0.014s
I20260709 12:15:21.438756 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:21.441270 23387 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:15:21.227157 28634 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.441390 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:21.135928 28574 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.431574 28626 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.427495 28624 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:21.464673 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:21.465593 23464 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:21.466336 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:21.467572 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:21.468277 23464 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:21.432857 28356 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:21.435425 28316 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.432929 28636 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:21.437466 28450 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:21.463747 28455 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.023s
I20260709 12:15:21.431293 28440 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:21.497874 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 902) took 1465 ms (client timeout 1719 ms). Trace:
I20260709 12:15:21.462827 28578 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.500957 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:21.438066 28609 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:21.527872 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:21.501186 23390 rpcz_store.cc:269] 0709 12:15:20.032254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.032383 (+   129us) service_pool.cc:224] Handling call
0709 12:15:21.494533 (+1462150us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54001e3f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:21.497799 (+  3266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":655437056}
I20260709 12:15:21.471393 28624 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.532850 24013 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:21.534605 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:21.535027 23463 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:21.536137 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:21.482167 28391 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:21.545593 24010 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 9: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:21.480631 28639 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:21.532286 28622 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:21.576678 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:21.477618 28626 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.435506 28042 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.572402 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:21.577749 23463 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:21.567560 23464 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:21.579310 28630 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:15:21.579988 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 1922) took 1359 ms. Trace:
W20260709 12:15:21.572937 28576 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.924s	user 0.011s	sys 0.002s
I20260709 12:15:21.615080 23312 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:21.615240 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:21.615504 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:21.614723 23315 rpcz_store.cc:276] 0709 12:15:20.220545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.220786 (+   241us) service_pool.cc:224] Handling call
0709 12:15:21.579782 (+1358996us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b5400389354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:21.579921 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9342720}
I20260709 12:15:21.580943 28646 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:21.582402 28594 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.527s	user 0.005s	sys 0.007s
I20260709 12:15:21.583212 28564 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.585368 24013 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:21.586033 28640 raft_consensus.cc:2764] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:21.485528 28316 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.587309 28508 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:21.612275 28557 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.004s	user 0.017s	sys 0.006s
I20260709 12:15:21.611533 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:21.580515 28637 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:21.502043 28455 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:21.581907 28644 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:15:21.618747 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:21.538143 28609 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:21.573734 28632 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.020s
I20260709 12:15:21.638146 23464 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.686074 23461 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:21.626219 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:21.620342 28373 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.781s	user 0.011s	sys 0.007s
I20260709 12:15:21.637898 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:21.698534 23312 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:21.656911 28558 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.000s	sys 0.014s
I20260709 12:15:21.561231 28643 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
W20260709 12:15:21.715677 28331 consensus_meta.cc:287] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.519s	user 0.006s	sys 0.003s
I20260709 12:15:21.716187 28331 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.718102 28634 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.718925 28455 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.624552 28636 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.720129 23461 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:15:21.721029 24010 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:21.727025 23464 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
W20260709 12:15:21.736626 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 430408us
I20260709 12:15:21.738596 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:21.695806 23469 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
W20260709 12:15:21.697809 28562 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.006s
I20260709 12:15:21.740443 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:21.622069 28576 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.643265 28557 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.741725 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:21.744076 28562 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.754257 28562 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.756516 28562 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:21.757422 28562 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:21.704049 23463 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:21.712034 28651 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.095s	user 0.005s	sys 0.002s
I20260709 12:15:21.625353 28646 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:21.705318 28558 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.000s	sys 0.015s
I20260709 12:15:21.744921 23464 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:21.736480 28643 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:21.765597 23393 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:21.762074 28635 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:21.743330 28585 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:15:21.783197 28473 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.668s	user 0.013s	sys 0.005s
I20260709 12:15:21.784255 23396 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:15:21.786046 28653 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:15:21.790261 28665 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:21.790805 28665 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:21.791896 28665 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.694165 28632 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.802114 23466 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:21.803367 28632 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:21.806494 28316 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.812790 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2457) took 1213 ms. Trace:
I20260709 12:15:21.813321 23462 rpcz_store.cc:276] 0709 12:15:20.599532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.984777 (+385245us) service_pool.cc:224] Handling call
0709 12:15:20.985071 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.062032 (+ 76961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.062072 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:21.087212 (+ 25140us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.394331 (+307119us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:21.453980 (+ 59649us) log.cc:844] Serialized 14 byte log entry
0709 12:15:21.502875 (+ 48895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.503053 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:21.812501 (+309448us) raft_consensus.cc:1685] finished
0709 12:15:21.812531 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.812707 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7652,"prepare.queue_time_us":59466,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":233675,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":15597,"thread_start_us":332190,"threads_started":3,"wal-append.queue_time_us":627138}
I20260709 12:15:21.818406 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:21.819360 28646 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:21.763300 28645 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.140s	user 0.000s	sys 0.006s
I20260709 12:15:21.694973 23315 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:21.769524 28440 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:21.824682 28503 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.941s	user 0.013s	sys 0.003s
I20260709 12:15:21.825547 28643 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.828408 28643 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:21.769899 28649 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:15:21.770049 28331 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.770453 23464 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:15:21.771482 28557 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.778367 23393 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:21.779969 28568 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:21.780592 28576 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:21.782457 28454 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.057s	user 0.012s	sys 0.005s
I20260709 12:15:21.831171 23462 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
W20260709 12:15:21.831776 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 392834us
I20260709 12:15:21.837337 28635 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:21.857242 23436 connection.cc:256] RPC call timeout handler was delayed by -1.0176695430000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
W20260709 12:15:21.840113 23361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.004s	sys 0.009s
W20260709 12:15:21.859433 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107047us
W20260709 12:15:21.860296 23436 connection.cc:256] RPC call timeout handler was delayed by -1.0176695430000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:15:21.825210 28503 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:21.694221 28560 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.879s	user 0.018s	sys 0.005s
W20260709 12:15:21.858799 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 420073us
I20260709 12:15:21.859938 23464 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:21.840866 28585 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:21.861861 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:21.868520 28658 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:15:21.865118 28632 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.870682 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2606) took 1063 ms. Trace:
I20260709 12:15:21.898411 23317 rpcz_store.cc:276] 0709 12:15:20.807361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.816118 (+  8757us) service_pool.cc:224] Handling call
0709 12:15:20.848791 (+ 32673us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.422526 (+573735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.870407 (+447881us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b5400296b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:21.870617 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317334272}
I20260709 12:15:21.875733 23395 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:21.842566 28643 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:15:21.877506 28560 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.878482 23321 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:21.879748 28113 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:21.879945 28503 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.881402 28630 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:15:21.865537 28316 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.883181 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:21.867132 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:21.885229 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1185) took 1341 ms. Trace:
I20260709 12:15:21.857077 23315 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:21.894269 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:21.909629 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45300.5us
W20260709 12:15:21.910503 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43508.6us
I20260709 12:15:21.906998 24011 rpcz_store.cc:276] 0709 12:15:20.543543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.616023 (+ 72480us) service_pool.cc:224] Handling call
0709 12:15:20.620975 (+  4952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:20.844643 (+223668us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:21.049900 (+205257us) log.cc:844] Serialized 14 byte log entry
0709 12:15:21.147463 (+ 97563us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:21.147517 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:21.147558 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.884886 (+737328us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400650ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:21.884954 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.885140 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38800,"prepare.queue_time_us":185315,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":7931,"raft.run_wall_time_us":9217,"spinlock_wait_cycles":605010048,"thread_start_us":247821,"threads_started":2,"wal-append.queue_time_us":136956}
I20260709 12:15:21.857831 28454 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.901013 28568 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:21.922164 23313 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:21.923481 28667 raft_consensus.cc:2764] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:21.923985 28670 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:21.903661 28632 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.938305 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2620) took 1007 ms. Trace:
I20260709 12:15:21.927544 28624 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.948485 23319 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:21.947885 23312 rpcz_store.cc:276] 0709 12:15:20.930881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.714177 (+783296us) service_pool.cc:224] Handling call
0709 12:15:21.714550 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.714888 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.714916 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:21.715409 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.862097 (+146688us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:21.862158 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.862255 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:21.938062 (+ 75807us) raft_consensus.cc:1685] finished
0709 12:15:21.938084 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:21.938237 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7590,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":7601,"thread_start_us":146235,"threads_started":1,"wal-append.queue_time_us":209727}
I20260709 12:15:21.951297 23319 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:21.914563 28643 consensus_queue.cc:1243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:21.915500 28385 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:21.949293 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1110) took 1457 ms. Trace:
I20260709 12:15:21.946156 23467 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:21.949945 24013 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:21.940349 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:21.962849 24014 rpcz_store.cc:276] 0709 12:15:20.491414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.491618 (+   204us) service_pool.cc:224] Handling call
0709 12:15:21.949035 (+1457417us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003cb754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:21.949226 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378547584}
I20260709 12:15:21.948551 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1186) took 1403 ms. Trace:
I20260709 12:15:21.943977 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 911) took 1119 ms. Trace:
I20260709 12:15:21.952215 23313 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:21.964695 24012 rpcz_store.cc:276] 0709 12:15:20.545154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.628798 (+ 83644us) service_pool.cc:224] Handling call
0709 12:15:20.629205 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.194155 (+564950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.948311 (+754156us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b540074dbd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:21.948486 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297473408}
I20260709 12:15:21.965679 23393 rpcz_store.cc:276] 0709 12:15:20.825016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.835128 (+ 10112us) service_pool.cc:224] Handling call
0709 12:15:21.943785 (+1108657us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54002d23d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:21.943909 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010173824}
I20260709 12:15:21.909797 28440 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.969383 28624 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.981484 23312 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:15:21.982280 23312 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:21.982682 23312 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:21.920094 28647 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:21.921319 28646 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.934356 28671 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:21.939767 28560 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.945572 28454 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:21.991783 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 77055us
I20260709 12:15:21.915514 28316 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.920426 28473 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 12:15:21.974205 28670 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.997982 28386 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.999557 28670 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:21.993508 28647 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:22.002704 28632 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.942806 28568 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:21.916407 28585 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:21.939952 28635 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.007031 28568 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.007181 28316 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:21.988533 28440 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.012642 28440 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.014132 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2448) took 1433 ms. Trace:
I20260709 12:15:22.014721 23467 rpcz_store.cc:276] 0709 12:15:20.580222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.580558 (+   336us) service_pool.cc:224] Handling call
0709 12:15:20.580887 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.963064 (+1382177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.013752 (+ 50688us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002a0dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.014038 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139262336}
I20260709 12:15:21.994516 28671 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.017168 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 856) took 1115 ms. Trace:
I20260709 12:15:22.017841 28671 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.017787 24013 rpcz_store.cc:276] 0709 12:15:20.901582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.634652 (+733070us) service_pool.cc:224] Handling call
0709 12:15:21.949622 (+314970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.963473 (+ 13851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.016867 (+ 53394us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54003cb750. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.017076 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718924544}
I20260709 12:15:21.992723 28646 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.022022 28646 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.004356 28647 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.023145 28665 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.022603 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1118) took 1158 ms. Trace:
I20260709 12:15:22.028733 24007 rpcz_store.cc:276] 0709 12:15:20.863775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.131557 (+267782us) service_pool.cc:224] Handling call
0709 12:15:21.131957 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.188167 (+ 56210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.022344 (+834177us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540063e7d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.022534 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127712512}
I20260709 12:15:22.036532 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1060) took 1435 ms. Trace:
I20260709 12:15:22.036976 23469 rpcz_store.cc:276] 0709 12:15:20.600655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.047030 (+446375us) service_pool.cc:224] Handling call
0709 12:15:21.047285 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.740140 (+692855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.036291 (+296151us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400186ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.036464 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490633984}
I20260709 12:15:22.054631 28592 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:22.055325 28592 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.062492 28596 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:22.063004 28596 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.063866 28596 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.065312 28626 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:22.098783 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2608) took 1290 ms. Trace:
I20260709 12:15:22.100361 23318 rpcz_store.cc:276] 0709 12:15:20.808011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.818831 (+ 10820us) service_pool.cc:224] Handling call
0709 12:15:20.857789 (+ 38958us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:20.858099 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:20.858126 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:20.858631 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.892240 (+1033609us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b18000d56a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:21.965902 (+ 73662us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:21.965945 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.966015 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:22.097397 (+131382us) raft_consensus.cc:1685] finished
0709 12:15:22.097525 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.098375 (+   850us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7268,"raft.queue_time_us":54027,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":49239,"spinlock_wait_cycles":17100672,"thread_start_us":73441,"threads_started":1,"wal-append.queue_time_us":1145554}
I20260709 12:15:22.104317 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.107342 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:22.107178 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 232915us
W20260709 12:15:22.114073 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 118983us
W20260709 12:15:22.114480 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 118352us
W20260709 12:15:22.115860 24009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1107) took 1630 ms (client timeout 1533 ms). Trace:
W20260709 12:15:22.118459 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 108391us
W20260709 12:15:22.116575 24009 rpcz_store.cc:269] 0709 12:15:20.485487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.485620 (+   133us) service_pool.cc:224] Handling call
0709 12:15:22.115020 (+1629400us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540074e854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.115415 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500217216}
I20260709 12:15:22.097232 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:22.133227 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.132927 28564 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.136619 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2046) took 1932 ms. Trace:
I20260709 12:15:22.138198 23395 rpcz_store.cc:276] 0709 12:15:20.188052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.188240 (+   188us) service_pool.cc:224] Handling call
0709 12:15:21.142826 (+954586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.899939 (+757113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.119726 (+219787us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540005ad54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.120426 (+   700us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b540005ad50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.120694 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3809973760}
I20260709 12:15:22.145941 28564 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.149507 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2619) took 1199 ms. Trace:
I20260709 12:15:22.150187 23316 rpcz_store.cc:276] 0709 12:15:20.914355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.264177 (+349822us) service_pool.cc:224] Handling call
0709 12:15:21.607771 (+343594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.608374 (+   603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.608417 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:21.623160 (+ 14743us) log.cc:844] Serialized 26 byte log entry
0709 12:15:21.841263 (+218103us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:15:21.841289 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:21.841335 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:21.871449 (+ 30114us) raft_consensus.cc:1685] finished
0709 12:15:21.871471 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.112934 (+241463us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400389350. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.113398 (+   464us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7376,"spinlock_wait_cycles":745314688,"thread_start_us":146258,"threads_started":1,"wal-append.queue_time_us":227332}
I20260709 12:15:22.155359 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1338) took 1349 ms. Trace:
I20260709 12:15:22.155892 23321 rpcz_store.cc:276] 0709 12:15:20.805653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.805818 (+   165us) service_pool.cc:224] Handling call
0709 12:15:20.806140 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.906636 (+1100496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.154732 (+248096us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003f9dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.155167 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2465735424}
I20260709 12:15:22.159222 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2463) took 1333 ms. Trace:
I20260709 12:15:22.162966 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 1065) took 1557 ms. Trace:
I20260709 12:15:22.163497 23470 rpcz_store.cc:276] 0709 12:15:20.605726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.806087 (+200361us) service_pool.cc:224] Handling call
0709 12:15:22.162433 (+1356346us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b54000e1d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.162674 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14337280}
I20260709 12:15:22.170817 28564 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:22.171303 23314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1333) took 1391 ms (client timeout 1823 ms). Trace:
W20260709 12:15:22.171785 23314 rpcz_store.cc:269] 0709 12:15:20.780038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.780184 (+   146us) service_pool.cc:224] Handling call
0709 12:15:22.170907 (+1390723us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54002751d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.171111 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410228608}
I20260709 12:15:22.153043 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.183372 23316 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:22.185561 28564 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.186431 28564 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.188200 23319 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:22.189010 28564 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.190148 23315 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:22.162863 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:22.205127 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 87788.6us
I20260709 12:15:22.209331 23314 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:22.205209 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.211524 23314 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:22.218206 28387 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.219552 28387 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.222138 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.228291 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2623) took 1247 ms. Trace:
I20260709 12:15:22.228842 23317 rpcz_store.cc:276] 0709 12:15:20.981188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.914319 (+933131us) service_pool.cc:224] Handling call
0709 12:15:21.924750 (+ 10431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:21.953195 (+ 28445us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:21.953553 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.110275 (+156722us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:22.111511 (+  1236us) log.cc:844] Serialized 26 byte log entry
0709 12:15:22.111771 (+   260us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:22.112279 (+   508us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.126685 (+ 14406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.131609 (+  4924us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:22.192216 (+ 60607us) raft_consensus.cc:1685] finished
0709 12:15:22.192246 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.228095 (+ 35849us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13293,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":113,"raft.queue_time_us":60111,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":3251,"spinlock_wait_cycles":7553920,"thread_start_us":155884,"threads_started":1,"wal-append.queue_time_us":78238}
I20260709 12:15:22.231299 28592 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.214507 23321 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:22.238138 28674 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:22.240000 28518 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:22.239845 28588 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.241806 23319 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:22.159775 23468 rpcz_store.cc:276] 0709 12:15:20.825064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.422212 (+597148us) service_pool.cc:224] Handling call
0709 12:15:21.438819 (+ 16607us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:21.641014 (+202195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:21.641046 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:21.641533 (+   487us) log.cc:844] Serialized 52 byte log entry
0709 12:15:21.765767 (+124234us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:21.821936 (+ 56169us) log.cc:844] Serialized 14 byte log entry
0709 12:15:21.822166 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.158221 (+336055us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.158305 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:22.158426 (+   121us) raft_consensus.cc:1685] finished
0709 12:15:22.158602 (+   176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.158912 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17758,"prepare.queue_time_us":55972,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":19899,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":48486,"spinlock_wait_cycles":422935296,"thread_start_us":54185,"threads_started":1,"wal-append.queue_time_us":182719}
I20260709 12:15:22.241957 28518 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:22.244593 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 115724us
I20260709 12:15:22.252384 28609 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:22.241724 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 1.00949e+06us
I20260709 12:15:22.257921 28663 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.258672 28084 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:22.258864 28663 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.260265 28351 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.262708 28611 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.263707 28611 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.265785 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.266120 28588 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:22.266379 28084 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:22.266924 28084 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:22.258701 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2622) took 1277 ms. Trace:
I20260709 12:15:22.269923 23315 rpcz_store.cc:276] 0709 12:15:20.980938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.914043 (+933105us) service_pool.cc:224] Handling call
0709 12:15:22.189587 (+275544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:22.206361 (+ 16774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.258409 (+ 52048us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002751d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.258622 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632759552}
I20260709 12:15:22.270679 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.271054 28596 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:22.271458 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.272346 24012 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:22.273312 24014 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:22.268939 28084 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.275624 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:22.256946 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:22.235693 28592 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:22.283687 28084 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:22.243638 28518 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.312182 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2460) took 1487 ms. Trace:
I20260709 12:15:22.301075 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.318157 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.318329 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.319880 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.301625 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.301903 28387 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.321705 24007 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:22.322091 28663 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.302393 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:22.302855 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:22.324615 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1190) took 1700 ms. Trace:
I20260709 12:15:22.324990 23396 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:22.303298 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50460.5us
I20260709 12:15:22.325143 24008 rpcz_store.cc:276] 0709 12:15:20.576623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.684320 (+107697us) service_pool.cc:224] Handling call
0709 12:15:20.684724 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:20.797670 (+112946us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:20.930134 (+132464us) log.cc:844] Serialized 14 byte log entry
0709 12:15:21.214498 (+284364us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.062970 (+848472us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b18003546a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1
0709 12:15:22.215465 (+152495us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b18003546a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:22.222721 (+  7256us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b18003546a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:22.223701 (+   980us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.224142 (+   441us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:22.224243 (+   101us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:22.224405 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.225336 (+   931us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540028f0d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.225380 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.277176 (+ 51796us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":991047,"prepare.queue_time_us":202153,"prepare.run_cpu_time_us":181,"prepare.run_wall_time_us":183,"raft.queue_time_us":1038167,"raft.run_cpu_time_us":528,"raft.run_wall_time_us":16356,"spinlock_wait_cycles":473646848,"thread_start_us":402543,"threads_started":2,"wal-append.queue_time_us":555645}
I20260709 12:15:22.287045 28682 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:15:22.331352 28494 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:22.331713 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1188) took 1780 ms. Trace:
I20260709 12:15:22.303848 28118 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:22.304327 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.322925 23466 rpcz_store.cc:276] 0709 12:15:20.824302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.133270 (+308968us) service_pool.cc:224] Handling call
0709 12:15:21.183613 (+ 50343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:21.802587 (+618974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.311609 (+509022us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b5400225bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:22.311964 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1421979904}
I20260709 12:15:22.333196 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.312274 24011 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:22.295348 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.334865 28325 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:22.336196 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.336535 28558 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:22.337095 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 80366.8us
I20260709 12:15:22.337328 28558 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.337966 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.338184 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.338374 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:22.338832 23466 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:22.338946 23469 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:22.340758 23468 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:22.341821 28049 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:22.342257 28049 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.342514 28558 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.342623 28049 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:22.343407 28558 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:22.344605 28391 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:22.344964 28558 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:22.345160 28391 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.345629 28609 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:22.345978 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.346932 28611 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.347251 28508 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.348078 28564 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:22.349021 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:22.337116 23470 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:15:22.356314 24007 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:22.352428 28558 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:22.368415 28391 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:22.386896 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76341.6us
I20260709 12:15:22.386546 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:22.332209 24016 rpcz_store.cc:276] 0709 12:15:20.551498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.634195 (+ 82697us) service_pool.cc:224] Handling call
0709 12:15:20.659634 (+ 25439us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:15:20.665948 (+  6314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:20.665979 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:15:20.696595 (+ 30616us) log.cc:844] Serialized 78 byte log entry
0709 12:15:20.839929 (+143334us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:21.009271 (+169342us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.051272 (+1042001us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b1800447460. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:22.142419 (+ 91147us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b1800447460. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:22.330569 (+188150us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.331042 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.331213 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:22.331313 (+   100us) raft_consensus.cc:1685] finished
0709 12:15:22.331331 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.331519 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218874,"prepare.queue_time_us":168688,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":2674,"raft.queue_time_us":35050,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":23804,"spinlock_wait_cycles":22793344,"thread_start_us":379477,"threads_started":3,"wal-append.queue_time_us":329991}
I20260709 12:15:22.389132 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.391019 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.391775 24007 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.392940 23461 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:22.397863 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:22.402670 23395 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:22.403831 23312 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.405835 23313 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:22.407794 23464 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:22.415102 23468 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:22.416530 23466 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:22.383404 28508 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.420257 28508 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.383625 28663 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:15:22.397593 28564 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.422410 23463 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
W20260709 12:15:22.425248 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45822us
W20260709 12:15:22.426764 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49222us
I20260709 12:15:22.381872 28611 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.430574 24007 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:22.436126 23312 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.441736 28437 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:15:22.443020 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.443775 28623 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.444540 28623 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.445742 28568 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.446278 28568 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.447075 28623 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.447675 28623 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:22.448343 28623 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.449148 28375 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:22.450070 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:22.433286 28387 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:15:22.451519 28681 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:22.451875 28681 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.453562 28674 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:22.455354 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.456375 23469 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:22.457161 28555 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:22.457667 28555 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.459113 28555 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.461714 28564 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.462514 28665 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:22.464910 28626 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:22.467779 28671 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:22.468250 28671 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.469800 28611 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:22.470350 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 155928us
I20260709 12:15:22.472299 28611 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.472585 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:22.461743 28560 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:22.476666 28560 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.477679 28508 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:22.478506 28508 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:22.443821 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 95042.2us
I20260709 12:15:22.472469 28558 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.448694 28568 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.485031 24009 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
W20260709 12:15:22.487845 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52369.3us
I20260709 12:15:22.462532 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2456) took 1864 ms. Trace:
I20260709 12:15:22.483460 24007 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 12:15:22.497502 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62562.6us
I20260709 12:15:22.484308 28387 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:22.500856 28387 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.502089 28387 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.503726 28387 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:22.503990 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.487172 28674 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:22.507520 28674 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.508277 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.481387 28508 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.508776 28674 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.507002 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.524066 23466 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:22.524914 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.452832 28681 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:22.526819 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 54381us
I20260709 12:15:22.429952 28609 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.527240 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.527699 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.493175 28671 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.495572 28626 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:22.481734 28522 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
W20260709 12:15:22.530095 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 96832.5us
I20260709 12:15:22.677736 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.517844 28674 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:22.495584 23465 rpcz_store.cc:276] 0709 12:15:20.598400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:20.965307 (+366907us) service_pool.cc:224] Handling call
0709 12:15:22.183824 (+1218517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:22.184455 (+   631us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:22.184986 (+   531us) log.cc:844] Serialized 14 byte log entry
0709 12:15:22.203172 (+ 18186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:22.203225 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:22.203308 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.203423 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.462149 (+258726us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000e1d50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:22.462314 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":916,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":137,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":233,"spinlock_wait_cycles":2758834560,"thread_start_us":17181,"threads_started":1,"wal-append.queue_time_us":17725}
I20260709 12:15:22.639608 23469 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:22.564617 23396 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:22.690407 23312 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:22.540170 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.688416 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.540786 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.639001 28042 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:15:22.698369 23470 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:22.698674 28648 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.711045 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:22.678289 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.517843 28508 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.545831 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.524188 28665 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:15:22.517447 28518 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:22.624926 28555 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:22.639883 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 112308us
I20260709 12:15:22.548789 28375 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:22.709268 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 180507us
I20260709 12:15:22.533385 28558 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:22.546478 28596 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:22.546741 28568 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.542528 28560 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.776925 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.775914 23315 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
W20260709 12:15:22.706151 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.678382 28611 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.539915 28681 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:22.779870 23466 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:22.548274 28454 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.781574 28648 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.782398 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.782514 24010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1193) took 1536 ms. Trace:
I20260709 12:15:22.783491 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:22.711778 28663 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:15:22.829530 28663 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:22.829890 28663 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:22.830842 28663 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.832368 28663 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:22.832741 28648 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:22.833705 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50294.1us
I20260709 12:15:22.833911 23320 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:22.834793 28691 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:22.834975 28117 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:22.835695 28117 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.835608 28691 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.791432 28325 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:22.836795 28325 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:22.837239 28325 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.836920 28117 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.793192 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.793607 28522 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:22.793907 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83022.6us
I20260709 12:15:22.798125 28623 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:22.798825 23313 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:22.798918 28671 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.696852 28049 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.846518 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.847177 24009 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:22.847914 28049 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.715574 28391 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.805902 23464 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:22.823002 28084 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.813115 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.777670 28665 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.640177 28437 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
I20260709 12:15:22.780663 28042 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:22.826128 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.711473 28494 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
I20260709 12:15:22.826823 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:22.819761 28596 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.830017 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.843731 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.529554 28385 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.316s
I20260709 12:15:22.844574 23466 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:22.873710 28665 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.874161 23317 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:22.827346 24010 rpcz_store.cc:276] 0709 12:15:20.962337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:21.736906 (+774569us) service_pool.cc:224] Handling call
0709 12:15:21.737273 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:22.255990 (+518717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:22.256035 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:22.256899 (+   864us) log.cc:844] Serialized 52 byte log entry
0709 12:15:22.427388 (+170489us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:22.427447 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:22.432817 (+  5370us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:22.490295 (+ 57478us) raft_consensus.cc:1685] finished
0709 12:15:22.490314 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:22.498433 (+  8119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":517978,"raft.queue_time_us":520499,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":156,"thread_start_us":144318,"threads_started":1,"wal-append.queue_time_us":228507}
I20260709 12:15:22.875052 28665 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:22.865087 28454 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.876410 28522 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.877599 23465 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:22.879455 28522 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.863025 28042 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:22.867060 28084 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.880854 28042 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.882092 28042 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.882849 28084 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:22.885543 23468 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:22.886500 24016 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:22.888046 24015 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:22.810846 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:22.892347 23470 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:22.867281 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.825924 28375 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.863054 28635 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:15:22.872627 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.705221 28626 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.874244 28391 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.849679 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.893834 23316 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:22.853883 28568 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.826861 28611 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:22.825541 28508 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:22.825169 28518 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:22.891431 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.906312 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.907604 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.911255 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.893858 28596 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:22.914001 23315 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:22.921435 23320 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:15:22.925150 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:22.928088 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:22.927064 23318 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 8.
I20260709 12:15:22.931609 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:22.935890 28494 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:22.936347 28494 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:22.936400 28635 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:22.936836 28494 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:22.941241 23313 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:22.942795 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.947137 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.952033 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:22.954284 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:22.955415 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.956537 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.959233 23396 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:22.936857 28635 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.962023 23318 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:22.961349 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.966259 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.962620 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:22.969563 28579 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.110s
I20260709 12:15:22.986037 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:22.987582 28579 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:22.998413 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.996270 23465 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:22.969812 28391 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:15:23.012030 23313 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:23.013089 23316 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:15:23.017213 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.019644 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:22.985510 28118 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:23.029021 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.030124 24016 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:22.990979 28691 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:22.992219 28375 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.039763 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:23.039707 28579 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.042639 28579 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.029212 28118 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:23.050724 28387 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:23.051226 28387 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:23.051283 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.052266 28387 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.054280 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.054991 23395 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:23.055925 28387 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.059643 28568 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.060313 28568 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:23.038204 28635 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.062594 28325 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.063541 28635 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.064965 28325 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.039110 28049 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:23.065640 28049 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:23.066869 28049 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.068353 28049 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.069891 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.053294 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.076512 28473 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:23.077536 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.077634 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.080621 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.085554 23316 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.088822 28454 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.089362 28546 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:15:23.090683 28546 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:23.091080 28546 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:22.985101 28518 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.045047 28118 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.093986 28118 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:23.099727 28508 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.076583 28704 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.100754 23315 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 7.
I20260709 12:15:23.101071 28704 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.092109 28546 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.111359 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.111392 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.115216 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.123009 23319 raft_consensus.cc:2393] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.124892 23312 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:23.125336 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:23.128309 23362 leader_election.cc:400] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:15:23.128765 23362 leader_election.cc:403] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:23.078953 28648 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:23.079887 28473 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.133419 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:23.135144 23315 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:23.140618 28596 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.142906 28596 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.146274 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.146682 28646 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:15:23.147230 24016 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:23.147609 28579 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:15:23.155607 24015 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.127080 28508 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.134085 28473 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:23.173310 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 120172us
I20260709 12:15:23.175422 28609 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.183250 23393 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:15:23.180207 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:23.191095 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110822us
I20260709 12:15:23.192258 24010 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.193580 24007 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:23.202152 28704 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.203329 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.205574 28508 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.206130 28508 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.208230 28508 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.215462 28084 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:23.216058 28084 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:23.217741 28674 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:23.223198 28546 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.229838 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:23.230461 23465 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
W20260709 12:15:23.234730 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.234879 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:23.235509 23461 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:23.241627 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:23.242470 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.244865 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.217196 28084 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.250638 23470 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:23.228965 23315 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
W20260709 12:15:23.253717 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.254801 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:23.255514 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:23.257545 28568 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:23.259351 28568 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:23.259846 28568 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:23.261706 28623 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:23.194834 28648 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.260771 28568 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.264660 28648 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.176824 28454 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.185611 28558 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:15:23.265384 28454 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.265679 28691 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:23.266450 28691 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:23.267490 28454 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.267599 28691 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.269650 23390 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:23.271382 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:23.274297 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46057.2us
I20260709 12:15:23.269656 28494 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.279629 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.280915 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.282505 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:23.283188 23317 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:23.285719 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.285917 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:23.286753 23470 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.287027 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.287694 23392 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:23.288455 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.289283 23391 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:23.290755 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:23.293555 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.294164 23392 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:23.271665 28681 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.301743 23387 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:23.301949 23395 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:23.326411 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.293941 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:23.327448 24007 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:23.337553 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:23.343917 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.346449 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:23.352190 28648 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.352970 28648 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.354337 23468 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:23.355850 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:23.356745 28648 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.361833 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:23.362673 23465 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:23.366611 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:23.367147 28558 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:23.367708 28558 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.368139 28558 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:23.368184 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:23.337136 28454 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.370208 28663 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:23.369925 28454 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.370651 28663 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:23.371111 28663 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:23.372814 28454 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:23.344089 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:23.373997 24011 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:23.374929 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.375830 28391 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.377506 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:23.377714 28681 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.382957 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.379885 24010 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:23.383185 28681 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:23.385066 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46585.6us
I20260709 12:15:23.385859 28391 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:23.386328 28391 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.386413 28522 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.386760 28391 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.386924 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.386927 28522 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.388386 28623 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:23.389598 28522 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.390223 28623 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:23.397950 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:23.405821 28681 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.285233 28473 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:23.419839 23461 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:23.389070 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.407058 28681 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.430681 23315 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:23.418143 28084 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:23.421440 28375 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:15:23.435560 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 141704us
I20260709 12:15:23.437316 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.417631 28494 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.428504 28454 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:23.444654 28049 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:23.445143 28375 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:23.445446 23321 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:23.432914 28665 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
I20260709 12:15:23.442956 28508 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:23.454387 28522 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:23.468940 23317 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:23.462560 28473 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.465718 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.471143 23315 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:23.467265 23465 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
W20260709 12:15:23.459479 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.478443 23396 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:23.479988 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:23.471292 28117 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.008s	sys 0.058s
W20260709 12:15:23.507843 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 118873us
I20260709 12:15:23.512195 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.513367 28623 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:23.513924 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.454387 28454 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:23.514932 28454 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:23.516067 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:23.517421 23392 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:15:23.482689 28704 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:15:23.519225 28704 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:23.519567 28704 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:23.519733 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.520527 28704 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.522595 28508 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:23.523975 28473 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.521232 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.514899 28626 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:23.519902 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:23.525408 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:23.510583 28117 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.009s	sys 0.058s
I20260709 12:15:23.524539 28473 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.538923 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.536351 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.471010 28391 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.538467 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.554935 24011 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:23.551218 23393 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:23.537082 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.565196 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:23.565558 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.649s (SENT) [suppressed 10 similar messages]
I20260709 12:15:23.566370 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.530668 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:23.561861 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:23.565217 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.572467 23393 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:23.584357 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.585068 23395 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:23.586216 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 55653.5us
I20260709 12:15:23.587321 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:23.548377 28648 raft_consensus.cc:2764] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:15:23.595105 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.537700 28623 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:23.526600 28611 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:23.623272 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:23.623790 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.625770 23317 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.621327 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:23.632484 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:23.632969 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:23.636857 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.638269 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.643517 23390 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:23.643795 23470 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:23.644165 24011 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:23.644640 23389 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:23.644973 23390 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:23.646008 28706 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:15:23.646082 23388 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:23.646250 28528 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:23.538645 28118 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:15:23.647717 28118 consensus_queue.cc:1243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:23.649053 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.649811 23318 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:23.652624 28391 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:23.658427 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 102048us
I20260709 12:15:23.643386 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.627933 23465 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:23.668301 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.670792 23316 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:23.618882 28674 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:23.622988 28546 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:15:23.688249 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 251023us
I20260709 12:15:23.689888 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:23.634168 28704 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:23.634377 28385 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:23.537806 28522 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.701354 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:23.705142 23391 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:23.723042 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 90683us
W20260709 12:15:23.706239 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 246072us
I20260709 12:15:23.538026 28626 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.700073 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:23.539101 28375 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.688308 28674 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:23.748281 23466 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:23.769631 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45820.9us
I20260709 12:15:23.770207 23394 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:23.621943 28117 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 12:15:23.771508 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:23.644670 28454 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:23.797461 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.696867 28681 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.769826 28648 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:23.786870 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:23.788831 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 223279us
I20260709 12:15:23.890781 23313 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.724152 28385 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:23.769309 28698 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:15:23.890902 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:23.895226 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:23.895810 23466 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.687829 28663 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:23.772891 28674 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:23.726219 28546 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:23.894883 23318 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:23.787915 28665 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:23.921839 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 405852us
I20260709 12:15:23.721510 28518 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.176s
W20260709 12:15:23.921562 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 220307us
I20260709 12:15:23.696075 28579 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.201s
I20260709 12:15:23.896320 28385 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:23.931030 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:23.936733 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:23.942456 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:23.956167 28707 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 12:15:23.958122 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:23.978367 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:23.939868 28084 consensus_meta.cc:287] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.002s
W20260709 12:15:23.981739 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 210404us
W20260709 12:15:23.983551 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 324108us
I20260709 12:15:23.939086 28473 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:23.943341 28647 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:23.999838 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:24.000535 23318 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
I20260709 12:15:24.003106 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:23.990053 28691 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:15:24.000595 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 51743.1us
I20260709 12:15:24.001366 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.015447 23465 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:24.016608 23466 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:24.003933 28049 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:24.017630 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.019470 23463 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:24.000780 28623 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:15:24.007401 28558 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
I20260709 12:15:24.004176 28391 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.025489 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:24.027698 28691 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:24.029424 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51125.2us
I20260709 12:15:24.029618 28049 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:24.018435 28647 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:24.004395 28611 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:15:24.036834 23318 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:24.038671 28626 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.038872 28049 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.018021 28546 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.041718 28454 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.043727 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:24.044914 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46701.7us
I20260709 12:15:24.035755 28623 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:24.047056 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.049479 28518 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:24.051499 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:24.055507 23392 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:24.057878 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.065395 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:24.063499 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.064749 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:24.069672 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:24.067211 23392 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:24.072139 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:24.073315 23466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:15:24.074667 23466 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:24.050885 28663 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.028445 28558 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.079217 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:24.086054 23394 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:24.090698 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:24.034291 28391 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.068173 28635 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:24.085889 28663 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:24.076711 28084 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:24.096606 23314 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:15:24.097157 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.071674 28721 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:24.078923 28518 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:24.090924 28647 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:24.102612 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.106892 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:24.107118 23982 leader_election.cc:336] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.696s (SENT)
I20260709 12:15:24.093184 28522 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.107939 23387 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:24.113129 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.123134 23465 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:24.108206 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:24.107630 24010 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:24.126152 23313 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:24.101920 28084 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.109480 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:24.110538 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.110973 28635 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:24.111334 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.096976 28682 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:24.113799 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.113147 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:24.130021 28084 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.102653 28518 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:24.144411 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.145800 24016 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:24.147560 23318 pending_rounds.cc:85] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:24.147910 23318 pending_rounds.cc:107] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:15:24.147943 28721 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:24.148319 23318 raft_consensus.cc:2889] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:24.127712 28691 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.139267 28648 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.150727 28648 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.138836 28673 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:24.154676 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.155592 28558 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.138926 28518 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.156734 23389 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:24.158267 28673 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.158957 28673 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:24.166025 28733 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:15:24.138835 28714 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:15:24.168200 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1734958689999075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:15:24.170017 23287 connection.cc:256] RPC call timeout handler was delayed by -1.154595868999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17930000000000001s for a response
W20260709 12:15:24.170449 28731 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:15:24.171088 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.173713 28742 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:24.174439 28522 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.174932 28742 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:24.175343 28742 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:24.175078 28522 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.176241 28742 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.138139 28647 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:24.177604 28648 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.178135 28647 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.169816 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.180744 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.156766 28724 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.187453 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.190603 23392 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:15:24.192667 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.195760 28522 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.196074 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.211659 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:24.196215 28729 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:15:24.197190 28648 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:24.197474 28391 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.200951 23463 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:24.202116 28385 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.137864 28623 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.199723 28518 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.183884 28555 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.225145 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.225587 28555 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:24.215204 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 108108us
I20260709 12:15:24.223225 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.212684 28742 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:24.230957 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.232496 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.233582 28555 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.233693 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:24.234346 23314 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:24.234658 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.218879 28522 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:15:24.235730 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.235924 28522 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:24.205238 28724 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.242910 23314 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:15:24.237109 28704 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.000s
I20260709 12:15:24.259479 28528 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:24.260058 28528 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.259856 28674 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.261705 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.229612 28648 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:24.259892 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:24.243930 28385 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.266052 23391 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 12:15:24.266811 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.274214 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.241680 28084 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.288226 28084 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:24.290206 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.292145 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.267580 28704 raft_consensus.cc:2764] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:24.306111 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2491) took 1078 ms. Trace:
W20260709 12:15:24.308188 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 96653.1us
I20260709 12:15:24.283778 28648 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:24.283239 28647 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:24.333747 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.338100 28385 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:24.338485 28518 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:24.338850 28385 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.341341 28385 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.341948 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.345484 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:24.346894 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.353812 28724 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.355225 28674 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:24.341959 28743 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:15:24.306788 23469 rpcz_store.cc:276] 0709 12:15:23.227938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.228499 (+   561us) service_pool.cc:224] Handling call
0709 12:15:23.237846 (+  9347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:23.238405 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:15:23.556821 (+318416us) log.cc:844] Serialized 14 byte log entry
0709 12:15:24.105598 (+548777us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:24.142792 (+ 37194us) log.cc:844] Serialized 52 byte log entry
0709 12:15:24.142985 (+   193us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:24.143219 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:15:24.305467 (+162248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:24.305606 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:24.305674 (+    68us) raft_consensus.cc:1685] finished
0709 12:15:24.305695 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:24.306010 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15382,"prepare.queue_time_us":550261,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":35502,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":4331,"thread_start_us":916759,"threads_started":4,"wal-append.queue_time_us":380226}
I20260709 12:15:24.325268 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.326406 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.371344 24016 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:24.372221 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 191610us
W20260709 12:15:24.373621 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.374477 28746 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:24.324123 28648 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.371973 28682 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:24.379914 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.382573 28724 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:24.383301 28724 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:24.385293 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.385600 28623 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:24.395156 28648 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:24.417711 24011 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:24.418393 28623 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:24.418448 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:24.397768 28681 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:24.419251 28681 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:24.419657 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:24.420063 28518 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:24.420413 28681 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.421653 28518 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:24.422060 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.422887 24016 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:24.400036 28508 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:24.405459 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:24.423595 28508 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:24.401043 28555 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.409826 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.410537 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.410972 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.435307 23387 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:24.411993 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:24.414333 23980 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:24.415097 24015 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:24.414994 28730 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:24.422163 28518 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:24.407811 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:24.432302 24010 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:24.440306 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:24.444869 24011 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:24.438179 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 114752us
I20260709 12:15:24.445727 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:24.446122 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:24.460594 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 198972us
I20260709 12:15:24.461470 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:24.465565 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 45982.8us
I20260709 12:15:24.435873 28623 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:24.469004 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.465086 28691 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:24.461987 28681 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:24.439642 28555 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.461079 28673 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:24.701637 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 255552us
I20260709 12:15:24.689386 28375 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:24.452917 28744 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:24.704319 23464 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:24.436108 28674 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
W20260709 12:15:24.706058 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 238823us
I20260709 12:15:24.464152 28635 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:24.424571 28508 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.469584 28732 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:15:24.710413 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:24.711862 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 247310us
I20260709 12:15:24.546555 28518 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.714457 28691 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:24.714679 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 249488us
W20260709 12:15:24.714784 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 237713us
I20260709 12:15:24.715040 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.715721 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:24.715835 28117 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:24.709004 28648 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:24.744683 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:24.745606 23466 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:24.746194 23462 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:24.747862 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:24.758015 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:24.729826 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:24.791471 23467 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:24.796783 23315 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:24.790295 23463 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:24.776145 23465 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:15:24.795603 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47602.8us
I20260709 12:15:24.777259 28722 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.730201 28673 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.731644 28555 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:24.731159 28528 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.819298 28635 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:24.834460 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:24.761668 28084 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.719558 28744 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:24.835204 23313 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
W20260709 12:15:24.808363 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 98057.4us
I20260709 12:15:24.826547 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:24.827042 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:24.790524 28717 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.769s	user 0.004s	sys 0.000s
I20260709 12:15:24.828935 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.764050 28742 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:15:24.761560 28647 raft_consensus.cc:2764] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:15:24.760805 28454 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:24.760536 28648 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:24.846381 28648 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:24.847271 28635 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.849677 28555 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.825645 23316 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:24.779640 28753 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:24.851861 23316 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:24.837152 23470 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
W20260709 12:15:24.838115 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 93604.6us
I20260709 12:15:24.769042 28760 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:24.819595 28042 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.714717 28759 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 12:15:24.838593 28385 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:24.881772 28385 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:24.883921 28385 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:24.757208 28626 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:15:24.853611 23465 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:24.886229 24007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:24.886826 24007 raft_consensus.cc:606] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:24.887156 24007 raft_consensus.cc:614] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:24.819432 28049 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:24.888805 28049 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.889364 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:24.890902 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:24.891014 28049 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:24.894388 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:24.897224 23464 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:24.912019 23465 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:24.841058 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:24.838578 28744 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:24.728555 28375 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:24.928994 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67788.5us
I20260709 12:15:24.819739 28494 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:15:24.854543 28508 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:24.840406 28742 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:24.830977 28722 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.923753 28753 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:24.986445 23462 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:24.925992 28385 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:24.853322 28084 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.003501 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.004606 23470 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:24.936597 28665 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:25.006906 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:25.009337 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 168360us
I20260709 12:15:24.923033 28042 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:24.990916 28742 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:25.011919 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:24.986362 28473 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:24.930289 28673 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:15:25.013322 23470 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.002116 28732 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.017071 28732 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.017984 28473 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.019691 28732 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.013337 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.020751 23462 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:25.021824 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.023175 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.023749 28555 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:25.024410 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.020704 28473 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.001482 28626 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:25.016309 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.030805 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.032405 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.017580 28722 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.035115 28528 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.038735 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.039918 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:24.894577 28760 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.006704 28049 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.025365 23462 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:25.040611 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 265282us
I20260709 12:15:24.925661 28691 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.004128 28568 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:15:25.043704 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:25.046841 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.047863 23464 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:15:25.048019 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:25.049609 23470 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:25.050545 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.053586 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.065479 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.042439 28673 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:25.072018 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.070050 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.073926 23462 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:25.075542 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.075574 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:25.076449 23470 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:25.078783 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.052388 28722 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:25.079684 23319 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
W20260709 12:15:25.072072 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 180111us
I20260709 12:15:25.067293 28732 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:25.107788 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.108942 23462 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:25.104278 23468 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:25.071456 28673 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:25.114638 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:25.047353 28528 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:15:25.070082 28724 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.116366 24015 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:25.046496 28049 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.118288 23468 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:15:25.075143 28518 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.080207 28626 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:25.121359 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 181975us
I20260709 12:15:25.127792 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:25.131723 23470 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:25.128525 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:25.135424 24011 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:25.112962 28759 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:25.135236 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:25.146245 24015 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:25.147426 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:25.154352 24015 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:25.127153 28722 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:25.131559 28732 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:25.175673 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 122213us
I20260709 12:15:25.178962 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:25.180094 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.192250 23470 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:15:25.233983 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53988.1us
I20260709 12:15:25.134150 28518 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.142931 28626 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.241268 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:25.134021 28724 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.182736 28759 raft_consensus.cc:2764] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:15:25.185775 28454 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:25.263679 28558 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.267143 28375 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:25.269196 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 134478us
I20260709 12:15:25.278012 28546 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:25.307865 28623 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.308079 28546 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:25.308503 28623 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.309018 28546 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.310288 28546 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:25.310771 28623 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.311311 28623 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:25.307904 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:25.242326 28387 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.238622 28635 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.121731 28753 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.240980 28084 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.323849 23312 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:25.286618 28042 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.330922 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:25.333062 23391 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:25.330574 28770 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.314613 28665 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.241123 28647 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:15:25.343837 28635 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.338697 28732 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.346438 28635 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:25.346719 28762 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:15:25.342343 28770 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.319492 28558 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.330143 28759 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:25.349859 28759 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:25.350076 28763 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:15:25.350606 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.328243 28528 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.350783 28759 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.351791 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.351550 28528 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.329906 28648 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.353578 23469 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:25.354601 28648 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.355229 28770 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.357805 28049 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.335098 28454 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.359542 28454 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.360687 28665 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.362852 28665 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.365729 28084 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.366339 28084 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.371945 28626 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.372319 28760 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.373191 28760 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:25.373739 28760 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:25.341375 28722 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.374630 28760 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.359006 28785 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:25.365903 28766 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:15:25.223698 28673 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.353219 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.386799 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:25.392617 24010 raft_consensus.cc:606] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:25.393028 24010 raft_consensus.cc:614] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:25.388477 28776 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:15:25.389261 28494 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.390185 28724 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:25.390430 28769 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:15:25.452034 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:25.387272 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 175076us
I20260709 12:15:25.330307 28387 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:25.414024 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60859.5us
I20260709 12:15:25.459700 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.369302 28084 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.466295 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.466962 23318 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:25.396970 28454 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.442447 28773 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.483718 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.459007 28674 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:25.495359 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.470355 28473 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:25.391574 28785 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.497964 28787 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:15:25.499872 28785 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.500574 28785 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:25.476611 28790 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:25.501811 28790 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.502235 28790 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:25.463467 28724 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:25.492605 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.493181 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.492304 28770 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:15:25.511807 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:25.484695 23316 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.505688 23318 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
W20260709 12:15:25.506690 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.487926 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.479195 28042 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.546124 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.501191 28785 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.452431 28494 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.551388 23316 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:25.572993 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.572793 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.574468 23318 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:25.510864 28118 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.523804 28117 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:25.582301 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 124872us
I20260709 12:15:25.587092 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.613899 23467 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:25.616457 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.617266 23313 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:25.618388 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:25.619302 23313 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:25.620270 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:25.622092 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 380886us
I20260709 12:15:25.576339 28647 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.504046 28674 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.648761 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:25.625123 23461 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:15:25.576756 28558 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:25.632917 28783 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.289s	user 0.003s	sys 0.000s
I20260709 12:15:25.640825 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.493896 28773 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.646646 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.627897 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.561141 28788 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:15:25.654740 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:25.656531 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.661465 23312 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:25.586937 28454 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.562140 28042 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.667552 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.577570 28785 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.654878 28724 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:25.665012 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:25.673661 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.683552 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:25.675827 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:25.684769 23470 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:25.669629 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.679527 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.679045 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.693403 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:25.692587 28793 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:25.697330 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.698186 23312 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:25.674472 28753 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:15:25.699190 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.699792 23312 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:25.700532 28773 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.700692 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:25.707623 28753 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:25.650110 28117 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.709872 23318 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 9.
W20260709 12:15:25.719563 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 206500us
I20260709 12:15:25.672066 28626 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.724525 23980 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:25.708312 28789 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:15:25.615130 28118 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.706101 28781 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:15:25.707226 28724 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:25.712589 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:25.706992 28785 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.728605 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.734066 23320 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:25.689581 28635 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:25.710265 23312 raft_consensus.cc:2376] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13. Re-sending same reply.
I20260709 12:15:25.653159 28647 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.692746 28767 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:25.689960 28648 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.709550 28779 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:25.674292 28759 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:15:25.740175 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:25.720710 28518 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.720906 28753 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:25.679011 28454 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:25.789621 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 107949us
I20260709 12:15:25.733870 28325 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:25.772200 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.764140 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.733503 28663 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:25.802506 23312 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:25.750958 28767 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.740908 28665 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:25.739948 28635 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.739866 28084 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:25.843818 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.796290 28387 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.850342 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.755708 28779 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:25.854967 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56522.8us
I20260709 12:15:25.856031 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:25.858441 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 124031us
I20260709 12:15:25.803846 28663 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.754262 28626 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:25.857708 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:25.878897 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.850095 28665 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:25.801604 28325 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.850090 28793 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.844947 28767 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:25.881762 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.798367 28648 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.888051 28807 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:25.850302 28635 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:25.851018 28084 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:25.888950 23434 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.649s (SENT)
I20260709 12:15:25.995827 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:25.888593 28117 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.915529 28626 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.858355 23314 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:26.012409 23396 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:26.013005 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.005594 28788 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:25.994812 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:25.979308 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.013916 28788 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.894735 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:25.895406 28792 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.167s
I20260709 12:15:26.015007 23391 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:25.926017 28793 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.014643 23217 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 11, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 11 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:25.953605 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:25.896201 28764 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:25.896589 28797 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:25.897392 28801 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:15:25.897538 28798 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 12:15:25.898394 28803 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:15:26.018371 28764 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.016547 23217 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 12 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:26.018852 28764 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:25.898801 28753 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.900830 28802 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:25.901846 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:25.902467 28778 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 12:15:26.019878 28802 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.903169 28674 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.904454 28118 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.905998 28805 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.021230 28805 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.022104 28802 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.023360 28805 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.995412 23395 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:25.890981 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:25.877533 28663 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:25.930437 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:26.014282 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.040454 23390 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:25.855695 28779 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:25.895642 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.016973 28797 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.027189 28673 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:26.048581 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 383624us
W20260709 12:15:26.028450 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:25.993152 24011 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:15:25.762629 28528 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:25.995692 24016 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:15:25.909356 28790 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:25.914065 28508 consensus_meta.cc:287] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.000s
I20260709 12:15:25.952844 24010 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:25.885291 28809 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:25.889925 28518 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:26.106858 28518 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.106750 28809 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.107457 28518 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.109155 28809 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:25.889443 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 96468us
I20260709 12:15:25.932283 28759 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:26.043907 23396 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:15:25.888439 28796 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.040153 23312 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:26.046921 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.127982 24009 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
W20260709 12:15:25.909303 28546 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.599s	user 0.008s	sys 0.002s
I20260709 12:15:25.878268 28791 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:25.996097 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:25.915241 28742 consensus_meta.cc:287] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.878s	user 0.009s	sys 0.001s
I20260709 12:15:26.134588 28742 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.135874 23466 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
W20260709 12:15:26.051783 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 162884us
I20260709 12:15:26.140407 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:25.891505 28665 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:25.912717 28744 consensus_meta.cc:287] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.923s	user 0.009s	sys 0.000s
I20260709 12:15:26.147687 28744 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.148876 24015 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:26.150022 28744 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.151232 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1086) took 1709 ms. Trace:
I20260709 12:15:26.151724 23469 rpcz_store.cc:276] 0709 12:15:24.441750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.441870 (+   120us) service_pool.cc:224] Handling call
0709 12:15:24.704793 (+262923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.353985 (+649192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.151002 (+797017us) spinlock_profiling.cc:243] Waited 649 ms on lock 0x7b54002da0d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:26.151173 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1491593472}
I20260709 12:15:25.892023 28325 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:25.949551 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.039815 28084 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:26.160238 28794 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:15:26.053342 28528 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:26.054031 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.054636 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.056458 28790 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.057986 28793 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.051980 23217 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:26.062644 28806 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.164s	user 0.002s	sys 0.000s
W20260709 12:15:26.066061 23214 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 220) took 31305 ms (client timeout 15000 ms). Trace:
I20260709 12:15:26.066501 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.172348 23314 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:26.068455 28814 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:15:26.070163 28788 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.001870 28647 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.072022 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:26.133278 28795 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.085503 23394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:26.069173 28781 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:26.086357 24011 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:26.088066 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:25.951071 28799 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.181869 28799 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.184151 28799 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.051424 28807 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.186224 28807 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:25.910796 28454 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:25.911183 28042 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.097004 28663 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.153630 28796 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.188560 28663 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:26.188815 28796 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.189504 23320 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:26.101897 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:26.189927 28811 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.306s	user 0.001s	sys 0.002s
I20260709 12:15:26.190217 23318 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:26.042906 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.109932 23390 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:26.191143 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.191939 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1231) took 1062 ms. Trace:
I20260709 12:15:26.043637 23316 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:15:26.192478 23316 pending_rounds.cc:85] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:26.192310 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.192749 23316 pending_rounds.cc:107] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:15:26.193004 23316 raft_consensus.cc:2889] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:26.102133 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:26.193701 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1390) took 1394 ms. Trace:
W20260709 12:15:26.193804 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91762.3us
W20260709 12:15:26.194111 28812 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.311s	user 0.002s	sys 0.000s
I20260709 12:15:26.194052 23316 rpcz_store.cc:276] 0709 12:15:24.799099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.799322 (+   223us) service_pool.cc:224] Handling call
0709 12:15:24.825539 (+ 26217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:25.439983 (+614444us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b4c00313100. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7940a20 00007ff0a7951a29 00007ff0a7917644 00007ff0a793ef2d 00007ff0a793eb93 00007ff0a966f18b 00007ff0a966fcb5 00007ff0a966af30 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416
0709 12:15:26.193451 (+753468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.193653 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17740032}
I20260709 12:15:26.111428 28797 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.131496 28555 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.141242 28792 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.195621 28555 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.197250 28795 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.197659 28781 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.199620 28770 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.130635 24010 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:26.199870 28781 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.200126 23317 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:15:26.200496 23317 pending_rounds.cc:85] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:15:26.200732 23317 pending_rounds.cc:107] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:15:26.200990 23317 raft_consensus.cc:2889] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:26.154865 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:26.200147 28770 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:26.202055 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47275.6us
I20260709 12:15:26.203298 28781 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:26.203720 28781 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.203799 28770 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.161949 23388 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:26.204125 28781 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:26.083925 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.170475 28809 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:15:26.205106 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 121333us
I20260709 12:15:26.130304 28385 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:15:26.206396 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.129757 28759 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.177917 23394 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:26.192349 24014 rpcz_store.cc:276] 0709 12:15:25.129401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:25.129574 (+   173us) service_pool.cc:224] Handling call
0709 12:15:25.129995 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:25.615121 (+485126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:25.615171 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:25.653319 (+ 38148us) log.cc:844] Serialized 52 byte log entry
0709 12:15:25.802034 (+148715us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:25.802091 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:25.802199 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:26.165742 (+363543us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:26.191633 (+ 25891us) raft_consensus.cc:1685] finished
0709 12:15:26.191658 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:26.191870 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24503,"raft.queue_time_us":40651,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":8464,"spinlock_wait_cycles":15867136,"thread_start_us":577813,"threads_started":2,"wal-append.queue_time_us":214947}
I20260709 12:15:26.156278 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:26.179297 23395 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:26.179729 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.181305 23389 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:26.016546 28767 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.210517 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.099372 28681 raft_consensus.cc:993] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:15:26.171790 23214 rpcz_store.cc:269] 0709 12:14:54.760853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:54.832181 (+ 71328us) service_pool.cc:224] Handling call
0709 12:15:26.065855 (+31233674us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:21.804388 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:21.805267 (+   879us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:21.805332 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:15:21.805358 (+    26us) tablet.cc:662] Decoding operations
0709 12:15:21.806935 (+  1577us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:21.807039 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:21.807091 (+    52us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:15:21.808666 (+  1575us) tablet.cc:701] Row locks acquired
0709 12:15:21.808727 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:15:21.808962 (+   235us) write_op.cc:270] Start()
0709 12:15:21.809182 (+   220us) write_op.cc:276] Timestamp: P: 1783599321808859 usec, L: 0
0709 12:15:21.809201 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:15:21.871830 (+ 62629us) log.cc:844] Serialized 5168 byte log entry
0709 12:15:22.051163 (+179333us) op_driver.cc:464] REPLICATION: finished
0709 12:15:22.060900 (+  9737us) write_op.cc:301] APPLY: starting
0709 12:15:22.061726 (+   826us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:22.071180 (+  9454us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:22.071333 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:22.619103 (+547770us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:22.790906 (+171803us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:15:22.790976 (+    70us) write_op.cc:312] APPLY: finished
0709 12:15:22.791599 (+   623us) log.cc:844] Serialized 121 byte log entry
0709 12:15:22.791833 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:22.792113 (+   280us) write_op.cc:454] Released schema lock
0709 12:15:22.792191 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:22.792770 (+   579us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:22.792801 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7706807,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":9445,"apply.run_cpu_time_us":23214,"apply.run_wall_time_us":731784,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":928105,"prepare.run_cpu_time_us":59789,"prepare.run_wall_time_us":224552,"raft.queue_time_us":22730,"raft.run_cpu_time_us":9179,"raft.run_wall_time_us":9781,"replication_time_us":241756,"thread_start_us":726600,"threads_started":4,"wal-append.queue_time_us":133237,"wal-append.run_cpu_time_us":5954,"wal-append.run_wall_time_us":1797560}]]}
I20260709 12:15:26.129051 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:26.211697 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 82792.5us
I20260709 12:15:26.212037 23315 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:26.175856 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 103919us
I20260709 12:15:26.212680 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:26.185736 28325 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:26.208803 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52603.9us
I20260709 12:15:26.209482 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.209594 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.208403 23312 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 9.
I20260709 12:15:26.185863 23217 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 8, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 8 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:26.195686 28792 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:25.909644 28724 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.137336 28742 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:26.166322 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 308690us
I20260709 12:15:26.214710 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.216116 23390 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:26.238109 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:26.239214 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1093) took 1465 ms. Trace:
I20260709 12:15:26.239624 23466 rpcz_store.cc:276] 0709 12:15:24.773803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.807744 (+ 33941us) service_pool.cc:224] Handling call
0709 12:15:24.833035 (+ 25291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.136269 (+1303234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.238948 (+102679us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540038ead4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:26.239154 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995816832}
I20260709 12:15:26.240350 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:26.240694 28808 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.355s	user 0.002s	sys 0.000s
I20260709 12:15:26.241998 23467 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:26.243070 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.243744 23389 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:26.244473 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:26.244735 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.245131 24010 raft_consensus.cc:606] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:26.170827 28635 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.218494 23388 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:26.217672 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:26.217757 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.225963 28797 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:26.219944 28787 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:26.180111 28744 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:26.215412 28796 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.215128 28788 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:26.222076 28809 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:26.235438 23314 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:26.246656 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:15:26.234800 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.245400 23389 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:26.245270 28753 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:26.233939 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:26.247136 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.275417 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.342151 23390 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:26.276201 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.343129 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:26.343624 23316 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:26.343732 23390 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:26.344841 23316 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:15:26.276608 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.346490 28767 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.346949 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.347558 23469 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:26.347433 28781 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.247327 23392 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:26.348429 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.349038 23469 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:26.245944 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:26.250002 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.254112 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1388) took 1459 ms. Trace:
I20260709 12:15:26.350679 23388 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:26.250483 23395 raft_consensus.cc:2393] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:26.350651 23317 rpcz_store.cc:276] 0709 12:15:24.795158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:24.795870 (+   712us) service_pool.cc:224] Handling call
0709 12:15:24.825152 (+ 29282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.201481 (+1376329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.253883 (+ 52402us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b540001de54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:26.254052 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3161367296}
I20260709 12:15:26.252787 28665 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.271660 28809 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.362677 28809 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:26.363300 28665 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.364349 23393 raft_consensus.cc:2368] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15.
I20260709 12:15:26.272500 28648 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.365885 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.366256 28042 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:26.367012 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.212244 28759 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:26.368054 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.257992 28663 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 12:15:26.273557 28788 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.370271 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.370609 28788 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:26.371078 23395 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:26.217245 28385 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:26.371922 28385 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:26.372344 28385 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:26.373864 24010 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:26.375064 28325 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.330770 23318 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:26.368098 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.377590 23314 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:26.334775 28764 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.267525 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.335292 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:26.387228 23461 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
W20260709 12:15:26.387619 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 52402.6us
I20260709 12:15:26.388535 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.335824 28387 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:26.336174 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.336920 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.405165 24009 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:26.337721 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.279695 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:26.338886 23319 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:15:26.339108 28635 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.411790 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:26.253976 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.339325 28807 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:26.340829 28792 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.278172 23312 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:26.275414 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:26.270872 23467 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:26.337721 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.418444 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:26.267339 28494 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.268015 28797 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:26.385660 28454 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:26.422313 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.422502 28797 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.263043 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.423113 28518 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:26.290568 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.267133 28795 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.361270 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:26.429567 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.365736 28820 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:26.333956 23217 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 4 to 8, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:26.379639 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:26.266232 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48616.9us
I20260709 12:15:26.433425 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:26.382429 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 148572us
I20260709 12:15:26.434544 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:26.266316 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.264825 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.435608 23392 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:26.436419 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.437569 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.439026 23463 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:26.368824 28779 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.383225 23980 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:26.440770 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 57633.7us
I20260709 12:15:26.441764 28732 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:26.441890 28779 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.442416 28732 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:26.442826 23391 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:26.308449 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 70247.3us
I20260709 12:15:26.443362 28732 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.447789 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:26.357414 23315 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:26.449891 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:26.450882 23315 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:26.451853 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.452486 23315 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.395238 23387 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:26.400620 23390 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:26.269660 23321 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:26.268002 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.401925 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.402598 23394 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:26.403417 23316 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:26.217355 28681 raft_consensus.cc:1081] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:26.413420 23396 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:26.273089 28555 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.423000 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:26.430227 23319 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:26.429567 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.477947 23461 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:26.479974 23316 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:26.354378 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.481060 23469 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:26.373095 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:26.270440 28744 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.268767 28787 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:26.489940 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56600.2us
I20260709 12:15:26.499518 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:26.490374 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 78715us
W20260709 12:15:26.499441 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 126433us
I20260709 12:15:26.494521 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:26.501062 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 264553us
I20260709 12:15:26.404224 28673 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:26.269160 28790 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:26.492688 23319 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
W20260709 12:15:26.413345 28736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.026s	sys 0.027s
I20260709 12:15:26.435353 24014 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:26.500007 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.407872 28767 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.516659 23316 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:26.516592 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:26.503363 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.503482 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.519724 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.504057 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:26.504525 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56825.3us
W20260709 12:15:26.498509 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64050.1us
I20260709 12:15:26.521926 23390 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:15:26.522462 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.524041 23389 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:26.505410 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.525925 23387 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:26.526924 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:26.502074 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:26.506204 23391 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:15:26.509053 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.512324 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:26.514478 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.425298 28518 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:26.532518 23319 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.270478 28084 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:26.570382 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53853.1us
W20260709 12:15:26.560662 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46296.2us
I20260709 12:15:26.571509 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.408223 28770 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:15:26.572351 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:26.566565 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44792.8us
I20260709 12:15:26.503952 28795 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:26.512876 28792 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:26.509815 28736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.027s	sys 0.027s
I20260709 12:15:26.510102 28781 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.508677 28555 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:26.499068 28117 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.574427 28375 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:26.504456 28744 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:26.606943 28787 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:26.586934 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:26.575241 28796 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:26.512341 28494 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:26.522837 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:26.662281 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 75470.1us
W20260709 12:15:26.662459 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 139835us
I20260709 12:15:26.663280 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:26.601666 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:26.623493 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 51208.3us
I20260709 12:15:26.432261 28807 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.523262 28767 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:26.559369 28724 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:26.576826 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55663.5us
I20260709 12:15:26.655982 23315 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:26.575345 28681 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:26.678881 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:26.681561 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:26.572746 23319 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:26.621601 28518 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.603881 28799 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.084s
I20260709 12:15:26.634140 28674 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:26.622805 28792 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.522680 28790 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.645992 28555 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.742967 28494 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:26.559851 28387 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:15:26.744411 23314 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:26.769105 23314 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:26.645052 28742 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.744598 24007 raft_consensus.cc:2393] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:26.721232 28821 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:26.659127 28626 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.751822 28787 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:26.751047 24016 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:26.660923 28375 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.755146 23315 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:26.790103 23315 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:26.721315 28795 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:26.748721 28546 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:15:26.740132 28118 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:26.740504 28767 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.766449 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:26.800760 23436 leader_election.cc:400] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15.
I20260709 12:15:26.801070 23436 leader_election.cc:403] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:26.745566 23217 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 8 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:26.719990 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:26.802520 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 138409us
I20260709 12:15:26.753393 28674 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.763284 28792 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:26.773473 28555 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:26.794628 23314 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.797633 23315 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.821309 23315 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.754061 28518 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.795495 28375 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:26.840457 23314 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:26.796296 28118 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:26.792301 28795 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:26.814950 28674 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:26.817317 28042 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:26.780896 28742 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.788636 28626 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.788333 28788 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.828117 28387 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:26.784307 28821 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.789345 28759 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.843250 28764 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.860299 28118 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:26.877900 23217 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 7, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 7 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:26.877846 23315 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:26.860159 28795 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:26.907984 23396 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:15:26.912040 23464 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:26.909161 28824 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 12:15:26.911096 28385 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 12:15:26.923235 28385 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:26.923944 28809 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.000s
I20260709 12:15:26.924371 28809 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.903456 28084 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.926613 28790 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.899010 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:15:26.928100 28759 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.928366 23315 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:26.928771 28759 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:15:26.928857 23315 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:26.919812 28821 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:26.922367 28518 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:26.930540 28742 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.931154 28742 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:26.931620 28742 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.932101 28807 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:26.934288 28118 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.909843 28387 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:26.938475 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.863332 28494 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:26.947242 23314 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:15:26.948074 23314 pending_rounds.cc:85] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:15:26.948402 23314 pending_rounds.cc:107] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 11.11
W20260709 12:15:26.949091 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:26.950078 23314 raft_consensus.cc:2889] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:26.958834 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.937640 28387 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:26.968708 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:26.974409 23315 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:27.019313 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.020129 23465 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:27.029314 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.030359 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:27.030539 23315 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:27.032011 24007 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:27.032660 24015 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:27.035387 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.036377 24015 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:27.083693 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.084457 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:27.086385 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.088430 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.089373 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.090045 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:27.022820 28732 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.019s	sys 0.043s
I20260709 12:15:27.088511 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.091624 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.087896 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.091548 24008 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:27.097986 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.101882 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:27.108507 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.042749 28742 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:27.094441 28732 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.019s	sys 0.043s
I20260709 12:15:27.119314 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.121037 28792 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:27.123463 28792 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:27.124279 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 1970) took 1443 ms. Trace:
I20260709 12:15:27.124747 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.125764 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:27.126564 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.133093 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:27.142812 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.143424 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:27.143613 23466 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:27.115399 28753 consensus_meta.cc:287] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.000s
I20260709 12:15:27.139349 23318 rpcz_store.cc:276] 0709 12:15:25.680613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.376758 (+696145us) service_pool.cc:224] Handling call
0709 12:15:26.377098 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.377474 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.377502 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.378007 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.878363 (+500356us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:15:26.878414 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.878479 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.124062 (+245583us) raft_consensus.cc:1685] finished
0709 12:15:27.124082 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.124222 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1728,"raft.queue_time_us":7650,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":16422,"thread_start_us":372987,"threads_started":1,"wal-append.queue_time_us":710658}
I20260709 12:15:27.112556 28626 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:27.168511 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.169255 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:27.171975 23470 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:27.131752 28809 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:27.180862 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:27.101257 28084 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:15:27.153733 28742 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.159722 28558 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:27.157904 28820 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:27.186678 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 386033us
I20260709 12:15:27.119906 28528 consensus_queue.cc:1059] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 12:15:27.187282 23287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.005s	sys 0.024s
W20260709 12:15:27.212352 23287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.005s	sys 0.024s
W20260709 12:15:27.212877 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 549677us
I20260709 12:15:27.213778 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.215749 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.154865 28799 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:15:27.224366 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.181545 28454 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:27.232396 23462 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:27.237363 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.209290 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1405) took 1282 ms. Trace:
I20260709 12:15:27.240790 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.195116 28558 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.259795 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:27.211268 28820 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:27.219805 28790 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:27.220211 28781 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:27.208204 28042 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:27.274820 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:27.276839 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 110431us
I20260709 12:15:27.265825 28558 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:27.217759 28742 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.307873 23466 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
W20260709 12:15:27.296402 28835 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.341s	user 0.002s	sys 0.000s
I20260709 12:15:27.245721 28848 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:27.243902 28764 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:15:27.240778 23320 rpcz_store.cc:276] 0709 12:15:25.926977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.463036 (+536059us) service_pool.cc:224] Handling call
0709 12:15:26.463414 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.463656 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:26.915046 (+451390us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.208897 (+293851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:27.208982 (+    85us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:27.209032 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.209104 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.209232 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":657244,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":86,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":13877,"thread_start_us":709351,"threads_started":2}
I20260709 12:15:27.267680 28753 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:27.297266 28084 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:27.275295 28820 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:27.323060 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2686) took 1319 ms. Trace:
I20260709 12:15:27.275609 28807 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:27.317163 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:27.295485 28781 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.305161 28518 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:27.294651 28842 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:15:27.325889 23313 rpcz_store.cc:276] 0709 12:15:26.004090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.510973 (+506883us) service_pool.cc:224] Handling call
0709 12:15:26.511263 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.655393 (+144130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.655419 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:26.655456 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:26.655489 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.322733 (+667244us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54004114d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:27.322802 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.322981 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330913664}
W20260709 12:15:27.273382 28797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.018s	sys 0.029s
W20260709 12:15:27.329144 28838 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
W20260709 12:15:27.347446 23434 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.825s (SENT)
I20260709 12:15:27.351161 28665 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.353379 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:27.357125 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.358278 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.358546 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:27.358788 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:27.354199 28790 raft_consensus.cc:2764] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:15:27.330282 28807 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:27.386718 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.364744 28795 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:27.321094 28084 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:27.333384 28781 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:27.361712 28847 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:15:27.411234 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.393954 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.343425 28518 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:27.396468 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.421168 23467 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:27.399189 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:27.401165 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.401958 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.424043 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.403357 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:27.356105 28797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.019s	sys 0.030s
I20260709 12:15:27.405897 23388 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:27.417893 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.425093 23394 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:27.426198 23390 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:15:27.426468 28843 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:15:27.428624 28827 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.550s	user 0.004s	sys 0.000s
I20260709 12:15:27.430485 28385 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:27.431756 28674 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.555s	user 0.004s	sys 0.004s
I20260709 12:15:27.432189 28674 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:27.433394 28826 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.566s	user 0.001s	sys 0.001s
W20260709 12:15:27.435313 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.436775 28851 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:15:27.439693 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.440838 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:27.381886 28494 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:15:27.411315 28845 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:27.442482 23462 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:27.445756 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.426328 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:27.446144 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.446524 23388 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:27.448693 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.449012 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.450435 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:27.437255 28759 consensus_meta.cc:287] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.000s
I20260709 12:15:27.452778 23314 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:15:27.393810 28807 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:27.451630 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:27.455227 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.455825 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:27.455753 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.412304 28084 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:27.460470 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.457381 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1415) took 1076 ms. Trace:
I20260709 12:15:27.458504 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.460930 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.461158 23313 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:27.461558 23316 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:27.459493 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.462618 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.461163 23319 rpcz_store.cc:276] 0709 12:15:26.381193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.749254 (+368061us) service_pool.cc:224] Handling call
0709 12:15:26.749651 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.752098 (+  2447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.752133 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.753028 (+   895us) log.cc:844] Serialized 26 byte log entry
0709 12:15:27.138372 (+385344us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:27.138436 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.138539 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.436226 (+297687us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:27.443440 (+  7214us) raft_consensus.cc:1685] finished
0709 12:15:27.443470 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.457315 (+ 13845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2025,"raft.queue_time_us":3234,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5583,"spinlock_wait_cycles":11029888,"thread_start_us":341044,"threads_started":1,"wal-append.queue_time_us":492329}
I20260709 12:15:27.483417 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.443563 28518 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.444192 28805 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:27.465723 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.393992 28665 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:27.468477 28846 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:15:27.494169 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:27.461923 28831 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 12:15:27.495918 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:27.453087 28759 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15.
I20260709 12:15:27.413353 28753 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:15:27.493511 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.445789 28842 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 12:15:27.494957 28805 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:27.411258 28042 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.483587 28790 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
W20260709 12:15:27.592061 28863 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
W20260709 12:15:27.605197 28847 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:15:27.613515 28862 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:15:27.641712 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1114) took 1368 ms. Trace:
I20260709 12:15:27.616536 28869 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:15:27.644496 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 148677us
I20260709 12:15:27.642175 23468 rpcz_store.cc:276] 0709 12:15:26.273757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.273877 (+   120us) service_pool.cc:224] Handling call
0709 12:15:26.274212 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.378598 (+104386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.378645 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.385667 (+  7022us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.843722 (+458055us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:26.847348 (+  3626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.933451 (+ 86103us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b54001874d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:26.933505 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.634933 (+701428us) raft_consensus.cc:1685] finished
0709 12:15:27.634962 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.641639 (+  6677us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36747,"raft.queue_time_us":43569,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"spinlock_wait_cycles":154604288,"thread_start_us":417999,"threads_started":1,"wal-append.queue_time_us":614610}
I20260709 12:15:27.620275 28787 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:27.654469 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2535) took 1226 ms. Trace:
I20260709 12:15:27.673192 23465 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:27.670467 23469 rpcz_store.cc:276] 0709 12:15:26.428190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.483406 (+ 55216us) service_pool.cc:224] Handling call
0709 12:15:26.483679 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.561717 (+ 78038us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.561747 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.575924 (+ 14177us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.979251 (+403327us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:27.014615 (+ 35364us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.014817 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.014935 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.654220 (+639285us) raft_consensus.cc:1685] finished
0709 12:15:27.654241 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.654388 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112546,"prepare.queue_time_us":39519,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":8428,"raft.queue_time_us":224921,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":46324,"thread_start_us":263653,"threads_started":1,"wal-append.queue_time_us":586784}
W20260709 12:15:27.677337 28855 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:15:27.682094 28732 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:27.682523 28732 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:27.684592 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.653244 28674 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.691246 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.691892 23312 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:27.648056 28871 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:27.688275 23315 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:27.689404 23464 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 1113) took 1416 ms (client timeout 1550 ms). Trace:
I20260709 12:15:27.689822 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.649252 28869 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:27.694554 23464 rpcz_store.cc:269] 0709 12:15:26.273381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.275056 (+  1675us) service_pool.cc:224] Handling call
0709 12:15:27.689215 (+1414159us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54000c7454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:27.689351 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136008960}
I20260709 12:15:27.697959 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:27.700475 23313 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:27.665738 28742 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.685066 28787 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:27.727847 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1404) took 1801 ms. Trace:
I20260709 12:15:27.683399 28732 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:27.731299 28872 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 12:15:27.732700 28860 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:15:27.737659 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2538) took 1057 ms. Trace:
I20260709 12:15:27.738211 23465 rpcz_store.cc:276] 0709 12:15:26.680652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.097407 (+416755us) service_pool.cc:224] Handling call
0709 12:15:27.737381 (+639974us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b5400226ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:27.737539 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1101518592}
I20260709 12:15:27.748292 23469 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:27.749694 23312 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:27.730239 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1243) took 1052 ms. Trace:
I20260709 12:15:27.750496 23313 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:15:27.750854 24014 rpcz_store.cc:276] 0709 12:15:26.678237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.678554 (+   317us) service_pool.cc:224] Handling call
0709 12:15:27.729966 (+1051412us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b540011e4d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:27.730165 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289920896}
I20260709 12:15:27.752345 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.733345 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:27.755060 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.760486 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:27.763654 23469 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:27.728441 23321 rpcz_store.cc:276] 0709 12:15:25.926707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.457155 (+530448us) service_pool.cc:224] Handling call
0709 12:15:26.457557 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.785956 (+328399us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:27.145149 (+359193us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.443237 (+298088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:27.443351 (+   114us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:27.443435 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.727517 (+284082us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54002751d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:27.727588 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.727764 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14124,"prepare.queue_time_us":335285,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":22211,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":19145,"spinlock_wait_cycles":761668864,"thread_start_us":637396,"threads_started":3,"wal-append.queue_time_us":311685}
I20260709 12:15:27.765102 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.715346 28787 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:27.795483 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 132177us
I20260709 12:15:27.796895 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.654031 28805 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.799156 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:27.766851 28846 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:15:27.804082 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.804858 23464 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:27.806152 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:27.764523 28797 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:27.810650 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:27.769096 28873 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:27.789759 28674 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:27.789042 28118 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:27.758994 28871 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:15:27.763737 28753 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 12:15:27.825467 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65450.5us
I20260709 12:15:27.830016 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2522) took 1447 ms. Trace:
I20260709 12:15:27.832355 23463 rpcz_store.cc:276] 0709 12:15:26.382760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.439735 (+ 56975us) service_pool.cc:224] Handling call
0709 12:15:26.440006 (+   271us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:26.513238 (+ 73232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.513273 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:26.559780 (+ 46507us) log.cc:844] Serialized 52 byte log entry
0709 12:15:27.095028 (+535248us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:27.125462 (+ 30434us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.154362 (+ 28900us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.154618 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.154731 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.829792 (+675061us) raft_consensus.cc:1685] finished
0709 12:15:27.829818 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:27.829961 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93887,"prepare.queue_time_us":25608,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":1716,"raft.queue_time_us":184471,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":96197,"spinlock_wait_cycles":13416320,"thread_start_us":378303,"threads_started":1,"wal-append.queue_time_us":618503}
I20260709 12:15:27.787779 28528 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:15:27.811050 28665 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.820888 28805 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.794118 28387 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:27.800631 28473 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:15:27.854064 28876 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:27.869035 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 118656us
W20260709 12:15:27.872018 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 73766.4us
I20260709 12:15:27.874687 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:27.878605 23464 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:15:27.879289 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80210.6us
I20260709 12:15:27.880254 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.871218 28874 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:15:27.875191 28744 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.152s	user 0.007s	sys 0.007s
I20260709 12:15:27.890368 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2090) took 1387 ms. Trace:
I20260709 12:15:27.809688 28869 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.898778 23396 rpcz_store.cc:276] 0709 12:15:26.502522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.502644 (+   122us) service_pool.cc:224] Handling call
0709 12:15:26.502971 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.908457 (+405486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:27.890062 (+981605us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54002c8b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:27.890301 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930888448}
I20260709 12:15:27.823969 28873 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.899685 28874 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:27.905931 28874 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.906672 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:27.889783 28473 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:27.911095 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:27.892441 28877 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:15:27.916263 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83780.3us
W20260709 12:15:27.909847 28849 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.211s	user 0.001s	sys 0.001s
I20260709 12:15:27.916626 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:27.920547 23315 raft_consensus.cc:2410] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 12:15:27.922263 28648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.030s	user 0.000s	sys 0.012s
W20260709 12:15:27.923009 28648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.000s	sys 0.012s
I20260709 12:15:27.866704 28787 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.912947 28674 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:27.901319 28744 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.914878 28869 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:27.918241 28870 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:27.934646 28781 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.939976 28781 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:27.919461 28797 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:27.918640 28877 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:27.925261 28473 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:27.909837 28878 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 12:15:27.944514 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:27.934331 28767 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.105s	user 0.008s	sys 0.005s
I20260709 12:15:27.948952 24013 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:27.947568 28876 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:15:27.955318 28876 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:27.932509 28753 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:15:27.956327 28807 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.957396 23318 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:27.958859 28744 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:27.944274 28883 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:27.961445 28797 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.931138 28528 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
W20260709 12:15:27.969810 28558 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.000s
W20260709 12:15:27.970261 28375 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.108s	user 0.011s	sys 0.003s
I20260709 12:15:27.970793 28375 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.973030 28375 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:27.973889 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.949069 28767 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.974604 28473 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:27.976984 28767 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.945529 28877 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:27.968061 28555 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.122s	user 0.013s	sys 0.000s
I20260709 12:15:27.977931 28555 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.980043 28555 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.969858 28873 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.981480 28084 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:27.983675 28788 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:27.988854 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:27.984359 28795 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:27.990615 23318 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:27.970358 28558 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:27.986451 28840 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.800s	user 0.009s	sys 0.002s
I20260709 12:15:27.965936 28883 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:28.009974 28883 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.012516 28883 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.992082 28848 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.013449 28848 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.016124 28848 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.016872 23319 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:28.017264 23319 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:27.999574 28558 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:28.000744 28870 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.002964 28084 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:28.003482 28807 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:27.997112 28674 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.035007 28874 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.028103 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1412) took 1724 ms. Trace:
I20260709 12:15:28.038017 23317 rpcz_store.cc:276] 0709 12:15:26.303624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.738566 (+434942us) service_pool.cc:224] Handling call
0709 12:15:26.756281 (+ 17715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.816339 (+ 60058us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:27.173130 (+356791us) log.cc:844] Serialized 14 byte log entry
0709 12:15:27.173441 (+   311us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:27.173507 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:27.173575 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:28.027822 (+854247us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b54003c0354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:28.027877 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:28.028031 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":305164,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":58,"raft.queue_time_us":4509,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":15688,"spinlock_wait_cycles":137558912,"thread_start_us":328884,"threads_started":1}
I20260709 12:15:28.045320 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 940) took 1814 ms. Trace:
I20260709 12:15:28.046057 23395 rpcz_store.cc:276] 0709 12:15:26.230959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.372579 (+141620us) service_pool.cc:224] Handling call
0709 12:15:26.372864 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:26.373231 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:26.373262 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:26.401789 (+ 28527us) log.cc:844] Serialized 26 byte log entry
0709 12:15:26.881617 (+479828us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:26.881655 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:26.881718 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:27.985912 (+1104194us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:28.036489 (+ 50577us) raft_consensus.cc:1685] finished
0709 12:15:28.036521 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:28.045252 (+  8731us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5802,"raft.queue_time_us":9370,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":16305,"spinlock_wait_cycles":51771776,"thread_start_us":377406,"threads_started":1,"wal-append.queue_time_us":673039}
I20260709 12:15:28.043412 23319 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.053901 23319 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.056771 28874 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.059845 28877 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:28.067615 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 122719us
I20260709 12:15:27.993520 28518 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.067091 28805 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.083527 23468 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:28.089562 28805 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.091231 28518 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:28.091886 28805 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:28.081571 28820 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.005s
I20260709 12:15:28.093293 28820 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.095667 28820 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:28.097810 28454 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:28.098343 28454 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:28.106451 24014 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:28.106734 28809 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.106889 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.107246 28809 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.108117 24007 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:15:28.106513 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:28.109617 28821 consensus_queue.cc:1059] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:28.109810 28877 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:28.107550 24015 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:28.111123 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:28.115684 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.116618 24015 raft_consensus.cc:2368] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
I20260709 12:15:28.126971 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.128367 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:28.128494 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.132314 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:28.131701 24007 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:28.133921 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.110474 28821 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:28.137193 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:28.109416 28809 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:28.138916 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:28.140226 23287 leader_election.cc:400] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
W20260709 12:15:28.140460 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 741326us
I20260709 12:15:28.140650 23287 leader_election.cc:403] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:15:28.140915 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:28.141242 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.132598 28805 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:28.143074 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 206384us
I20260709 12:15:28.132761 28674 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:15:28.139132 28821 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.137768 28854 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:28.153599 28854 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:28.153790 28792 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.154357 28792 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.154613 28854 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.155861 28854 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:28.156525 28792 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.157272 28792 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:28.139940 28870 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:28.136869 28117 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:28.138409 28874 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:15:28.152237 28042 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.175189 28042 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.152653 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2531) took 1725 ms. Trace:
I20260709 12:15:28.143961 28809 raft_consensus.cc:2764] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:15:28.142961 28807 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
I20260709 12:15:28.153471 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:28.177592 23461 rpcz_store.cc:276] 0709 12:15:26.427157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.479719 (+ 52562us) service_pool.cc:224] Handling call
0709 12:15:26.479974 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:26.480175 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:15:26.806564 (+326389us) log.cc:844] Serialized 14 byte log entry
0709 12:15:28.110909 (+1304345us) log.cc:844] Serialized 14 byte log entry
0709 12:15:28.152315 (+ 41406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:28.152368 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:28.152419 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:28.152482 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:28.152595 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":906495,"prepare.queue_time_us":288421,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":6522,"raft.queue_time_us":936298,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":4636416,"thread_start_us":764354,"threads_started":2,"wal-append.queue_time_us":514424}
I20260709 12:15:28.143128 28882 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:28.181674 28635 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:28.181865 28809 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:28.182121 28635 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:28.182966 28635 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.183583 28809 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:28.183947 28635 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:28.181722 28882 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.185864 28882 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.186733 28454 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.189759 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.190646 28856 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:28.191416 28842 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:28.192113 28375 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:28.193480 28042 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:28.194414 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.159217 28883 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.202387 28454 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.202977 28042 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.160712 28894 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:15:28.205260 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:28.205821 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.206125 28807 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:28.206269 28805 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:28.158495 28821 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:28.208345 28732 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:28.211023 28665 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:28.211412 28665 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:28.211848 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.212436 28665 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.213709 28884 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.214223 28884 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.216418 28665 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:28.216884 28884 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.157661 28792 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.218271 28792 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:28.141921 28895 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:15:28.224918 28877 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:28.225401 28877 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:28.226583 28877 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.199805 23982 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:28.484925 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 285179us
I20260709 12:15:28.486414 28888 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:28.486486 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:28.202664 28883 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.166642 28648 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:28.629340 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.630050 23468 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:28.160919 28325 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:15:28.166110 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:28.638104 28790 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:15:28.135768 28788 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 12:15:28.148356 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:28.145255 28473 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:28.251959 28807 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:28.157793 28870 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:28.711293 28870 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.160951 28874 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:28.716647 28874 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:28.256219 28877 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.252964 28882 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:28.805848 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.806396 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.807538 28882 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.425936 28895 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:28.458673 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.808287 28895 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.810458 28895 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.811131 23392 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:15:28.470595 28848 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:28.811933 28848 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.474242 28787 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.476487 28871 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:28.478485 28821 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.813792 28871 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.815191 28821 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.480655 23393 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:28.815871 28871 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.481119 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:28.488750 28883 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:28.490018 28894 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:28.818069 28894 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:28.819054 28894 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.492642 28888 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.503679 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:28.513422 28865 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.992s	user 0.000s	sys 0.005s
I20260709 12:15:28.825915 23394 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:28.544368 28042 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.559525 28792 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:28.570883 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.571784 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.828186 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.573467 28665 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:28.828886 28792 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.829313 28665 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.831429 28665 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.575147 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:28.833338 23387 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:28.576347 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:28.585011 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.590402 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:28.838229 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.839192 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.597986 28736 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:28.598706 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 393488us
I20260709 12:15:28.598799 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:28.613397 23469 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 7. Current term is 8. Ops: []
I20260709 12:15:28.841301 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.624661 28635 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:28.625478 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:28.629530 28648 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.635360 23461 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:28.848721 23315 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:28.185123 28809 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:28.641739 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 493428us
I20260709 12:15:28.641848 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.877957 23463 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:28.642720 28788 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.635393 28325 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:28.883129 24007 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:28.650985 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.885994 23464 raft_consensus.cc:2368] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15.
I20260709 12:15:28.651907 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.887019 23470 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.657811 23468 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:28.672835 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:28.691910 28673 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:28.697535 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 503158us
I20260709 12:15:28.718467 28874 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:28.720117 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:28.721504 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 555435us
I20260709 12:15:28.723909 28859 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:28.779855 28870 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:28.784143 24013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1144) took 1754 ms (client timeout 1504 ms). Trace:
I20260709 12:15:28.842125 23395 raft_consensus.cc:2393] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:28.833077 28888 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.844193 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.640095 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.852322 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.840765 28871 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:28.843791 28807 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:28.849985 28648 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:28.881575 24014 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:28.849833 28635 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.895846 24012 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:28.898231 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.937520 24008 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:28.898654 24015 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:28.899878 28842 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:28.901094 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.939632 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.940706 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.902493 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.941987 24010 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:28.903215 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.903888 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.914389 23463 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:28.906404 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.915838 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:28.943810 24011 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:28.905489 28673 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:28.915100 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.920940 23314 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:28.951385 23467 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:28.922585 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.854548 28790 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:15:28.924252 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.920650 23465 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:15:28.919359 28874 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.905242 23218 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:28.931972 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:28.933645 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89519.8us
I20260709 12:15:28.966897 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:28.967510 23464 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:15:28.919958 24013 rpcz_store.cc:269] 0709 12:15:27.029976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:27.042791 (+ 12815us) service_pool.cc:224] Handling call
0709 12:15:28.783907 (+1741116us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54006bdad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:28.784065 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952495872}
I20260709 12:15:28.933756 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.888273 23396 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:28.933768 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:28.970867 23468 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:28.944100 24007 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:28.849202 28736 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:28.950266 23316 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:28.941411 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.973632 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:28.865458 28880 consensus_queue.cc:1059] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:15:28.981319 24014 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:28.955554 23463 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:28.951145 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:28.984458 24015 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:28.985743 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:28.929212 23470 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:28.895211 28546 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:15:28.962591 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 328) took 26339 ms (client timeout 15000 ms). Trace:
I20260709 12:15:28.989821 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:28.990453 23466 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:28.962908 28870 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:28.918479 28859 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:28.914204 28845 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:15:28.929966 28788 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:29.012064 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:28.979416 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:28.988175 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.012715 24015 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.012871 23395 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:28.934525 28895 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:29.016924 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:29.017692 24013 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:29.018062 24010 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:15:29.018167 24016 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:29.018596 24016 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.011947 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:29.021934 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:29.023005 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.023506 23391 raft_consensus.cc:2376] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16. Re-sending same reply.
I20260709 12:15:29.024245 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:28.939850 28042 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:15:28.996682 23320 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:15:28.997900 23218 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:28.995510 23465 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:29.000247 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:29.034612 23317 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:15:28.999457 23316 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:28.930148 28854 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:15:29.027673 23395 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:29.019150 23314 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:28.935284 28871 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:28.999806 28807 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:28.989941 28875 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:29.029850 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.037050 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.037645 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.053143 23395 raft_consensus.cc:2368] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16.
I20260709 12:15:29.036005 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.010547 28859 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:28.930889 28821 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:15:29.090523 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.091390 23390 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:28.996987 28753 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:28.997572 28790 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.044417 23465 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:29.100628 23465 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:29.044916 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.049402 23387 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:29.048698 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.034500 23218 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 11, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 11 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:29.101269 23388 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:29.104662 23218 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 11 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 14 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:29.055096 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 139322us
I20260709 12:15:29.065846 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.092231 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.106925 23218 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:15:28.990206 23217 rpcz_store.cc:269] 0709 12:15:02.623245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.623593 (+   348us) service_pool.cc:224] Handling call
0709 12:15:03.002716 (+379123us) spinlock_profiling.cc:243] Waited 5.21 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:28.962498 (+25959782us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:27.699405 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:27.744549 (+ 45144us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:27.744613 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:15:27.744635 (+    22us) tablet.cc:662] Decoding operations
0709 12:15:27.745511 (+   876us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:27.745597 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:27.745623 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:15:27.746313 (+   690us) tablet.cc:701] Row locks acquired
0709 12:15:27.746346 (+    33us) write_op.cc:260] PREPARE: finished
0709 12:15:27.746514 (+   168us) write_op.cc:270] Start()
0709 12:15:27.746656 (+   142us) write_op.cc:276] Timestamp: P: 1783599327746449 usec, L: 0
0709 12:15:27.746680 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:15:27.812540 (+ 65860us) log.cc:844] Serialized 2936 byte log entry
0709 12:15:28.091733 (+279193us) op_driver.cc:464] REPLICATION: finished
0709 12:15:28.171122 (+ 79389us) write_op.cc:301] APPLY: starting
0709 12:15:28.171239 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:28.192414 (+ 21175us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:28.192574 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:28.248073 (+ 55499us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:28.248341 (+   268us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 12:15:28.248392 (+    51us) write_op.cc:312] APPLY: finished
0709 12:15:28.248770 (+   378us) log.cc:844] Serialized 73 byte log entry
0709 12:15:28.251071 (+  2301us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:28.251234 (+   163us) write_op.cc:454] Released schema lock
0709 12:15:28.251294 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:28.251758 (+   464us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:28.251783 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":165937,"reported_tablets":47,"spinlock_wait_cycles":11987328,"child_traces":[["op",{"apply.queue_time_us":79127,"apply.run_cpu_time_us":11663,"apply.run_wall_time_us":80608,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33415,"num_ops":7,"prepare.queue_time_us":488347,"prepare.run_cpu_time_us":16302,"prepare.run_wall_time_us":247269,"raft.queue_time_us":18080,"raft.run_cpu_time_us":19534,"raft.run_wall_time_us":74006,"replication_time_us":334698,"spinlock_wait_cycles":759552,"thread_start_us":240412,"threads_started":4,"wal-append.queue_time_us":175187}]]}
I20260709 12:15:29.053396 23466 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
W20260709 12:15:29.092181 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.111528 23218 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 9, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 9 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:15:29.117798 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 195260us
I20260709 12:15:29.118115 23388 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:15:29.119408 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 84605.3us
I20260709 12:15:29.119447 23391 raft_consensus.cc:2368] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19.
I20260709 12:15:29.123236 23388 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:29.125993 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:29.127125 23466 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:29.115509 23218 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 7, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 7 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.129966 23218 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.101461 23465 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.131717 23390 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:15:29.129256 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107701us
I20260709 12:15:29.099408 28753 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:29.162197 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:29.162220 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:29.163415 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.163815 23392 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:29.164469 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:29.041550 28878 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:29.041301 28848 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:29.135836 23436 connection.cc:256] RPC call timeout handler was delayed by -1.3341056640001625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260709 12:15:29.051957 28875 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:29.143013 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:29.064484 28792 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:29.027279 28788 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.229264 23461 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:29.032387 28895 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.143307 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.154306 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.166316 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.164978 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:29.161870 28753 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:29.135515 28821 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:29.216970 23388 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 12:15:29.218164 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.143241 28923 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:29.219197 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58297.2us
I20260709 12:15:29.236265 23218 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:15:29.219436 23287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.620s (SENT) [suppressed 14 similar messages]
I20260709 12:15:29.224557 23321 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:15:29.131436 28790 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.245568 23390 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:29.236243 23394 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:29.249550 24016 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:15:29.237161 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.241600 23393 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:29.236635 23387 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:29.132418 28673 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.254272 23218 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 13 to 14, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 14 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.272367 23218 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 15. New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:29.274108 23218 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:29.275362 23218 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.280534 23218 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 13, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.282209 23218 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.283535 23218 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 11, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:29.241078 28916 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 12:15:29.228900 28876 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.244133 28888 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:29.229565 28792 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:29.309650 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.293321 23317 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:29.293530 23218 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 11, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 11 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.311589 23218 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.251215 28923 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:29.317112 23218 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 13, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:29.319200 23218 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 7 to 10, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.321358 23218 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:29.331046 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 111627us
I20260709 12:15:29.324159 23218 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:29.235675 28929 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:15:29.335196 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:29.335705 24008 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:29.338263 23218 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.310246 28635 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.343384 23218 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:29.315419 28792 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:29.344779 23218 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:29.346038 23218 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:15:29.335829 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 69029us
I20260709 12:15:29.351946 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:29.308218 28673 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:29.321548 28933 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:15:29.361325 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 242494us
I20260709 12:15:29.362490 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:29.332767 28473 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.337814 28870 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.337970 28787 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.338143 28848 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.340008 28877 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.343978 28821 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.321801 28854 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:15:29.376230 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:29.394366 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.360752 28933 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:29.366241 28736 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:29.342943 28923 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:29.376783 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 445567us
W20260709 12:15:29.376626 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 367656us
W20260709 12:15:29.415671 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 49776.2us
I20260709 12:15:29.385955 28870 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.463861 24015 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
W20260709 12:15:29.465181 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 102813us
I20260709 12:15:29.322804 28876 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.408571 28949 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:15:29.428138 28919 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 12:15:29.432169 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800033e20 after lost signal to thread 22110
I20260709 12:15:29.343663 28925 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:15:29.385326 28787 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.414659 28934 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:15:29.401453 28877 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.313824 28875 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.375345 28932 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:29.379274 28871 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:29.435504 28788 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.378739 28926 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.406920 28821 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.405534 28848 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.452322 28790 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:29.449244 28325 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.462785 23982 leader_election.cc:400] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15.
I20260709 12:15:29.605365 23982 leader_election.cc:403] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:15:29.465322 23467 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 12:15:29.383183 28473 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:29.468784 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800351660 after lost signal to thread 23253
W20260709 12:15:29.612360 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9640 after lost signal to thread 23345
I20260709 12:15:29.466262 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:29.467124 23312 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.005s
W20260709 12:15:29.467662 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.498196 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 378082us
W20260709 12:15:29.503754 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.616031 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.616197 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:29.613045 23312 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.010s	sys 0.006s
I20260709 12:15:29.557801 28875 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:29.629848 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080029ace0 after lost signal to thread 23346
W20260709 12:15:29.631050 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080025bfe0 after lost signal to thread 23420
W20260709 12:15:29.632126 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08003902e0 after lost signal to thread 23494
W20260709 12:15:29.632382 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08003e6fa0 after lost signal to thread 23495
W20260709 12:15:29.599977 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.497673 28799 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.004s	sys 0.013s
I20260709 12:15:29.498557 28932 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:29.638890 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 265144us
I20260709 12:15:29.498095 28742 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:29.605984 23467 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:29.504525 28923 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:29.532272 28871 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:29.552753 28854 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:15:29.628448 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 170504us
I20260709 12:15:29.596318 28790 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.653961 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:29.635941 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800397b60 after lost signal to thread 24040
I20260709 12:15:29.665365 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:29.654047 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:29.637334 28799 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.005s	sys 0.016s
I20260709 12:15:29.636719 28635 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:29.708515 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:29.606786 28809 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.016s	sys 0.025s
I20260709 12:15:29.666899 23319 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:29.698591 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44736.7us
I20260709 12:15:29.692229 28882 raft_consensus.cc:2764] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:15:29.653512 28854 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:29.717854 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:29.553041 28926 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.702778 28934 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:29.713205 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 246997us
I20260709 12:15:29.736639 28934 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:29.724094 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.702418 28764 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:29.679442 28922 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:29.725462 28809 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.017s	sys 0.030s
I20260709 12:15:29.737062 28799 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:29.711355 28742 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:15:29.718452 28953 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.753001 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:29.752869 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:29.753413 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:29.723824 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:29.488130 28944 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:15:29.687222 28790 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:29.782138 28941 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:29.783273 28876 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:29.783815 28753 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.007s
W20260709 12:15:29.767939 28943 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:15:29.692348 28948 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.094s
I20260709 12:15:29.790869 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:29.766577 28958 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:29.754719 28882 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:29.815688 23318 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:29.800845 28942 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:29.822499 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:29.791782 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 63216.8us
I20260709 12:15:29.816073 28742 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:29.826083 28958 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.829294 28926 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.832885 23389 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:29.835163 28934 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.818449 28799 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:29.803649 28953 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:29.787839 28648 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.837s	user 0.006s	sys 0.006s
I20260709 12:15:29.847877 28764 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15.
I20260709 12:15:29.847803 28884 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:29.803838 28854 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.805717 28809 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:43504 (request call id 13) took 6250 ms. Trace:
I20260709 12:15:29.823627 28944 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:29.849447 23318 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:29.871642 28959 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.854156 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:29.864624 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:29.895871 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:29.817481 28821 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:29.851110 28871 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:29.788693 28922 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.877158 23314 pending_rounds.cc:85] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
W20260709 12:15:29.900888 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46821.4us
I20260709 12:15:29.902067 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:29.865906 28742 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:29.854668 28949 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:29.881479 28951 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:15:29.843911 28753 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.843214 28882 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:29.878275 28884 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:29.876811 28648 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.886650 28854 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:29.880203 28809 rpcz_store.cc:276] 0709 12:15:23.542063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:23.542202 (+   139us) service_pool.cc:224] Handling call
0709 12:15:23.692458 (+150256us) log.cc:844] Serialized 508 byte log entry
0709 12:15:24.374463 (+682005us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b540017ee20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ed142 00007ff0a95cdeaa 00007ff0a95d4d53 00007ff0a95d4cd9 00007ff0a95d4c69 00007ff0a95d4c31 00007ff0a95d3dfd 0000000000388634 0000000000388479 00007ff0a6587475
0709 12:15:29.792096 (+5417633us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":186,"mutex_wait_us":1930,"raft.queue_time_us":6053,"raft.run_cpu_time_us":4304,"raft.run_wall_time_us":427885,"spinlock_wait_cycles":16993152,"thread_start_us":208840,"threads_started":1,"wal-append.queue_time_us":287204,"wal-append.run_cpu_time_us":2928,"wal-append.run_wall_time_us":1153742}
I20260709 12:15:29.889065 28953 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.903671 28944 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:29.889707 28958 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:29.874025 28926 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:29.931082 23318 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:29.878113 28963 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.873906 28934 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:29.898427 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1438) took 1051 ms. Trace:
I20260709 12:15:29.898761 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:29.884100 28876 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.901050 23314 pending_rounds.cc:107] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:15:29.978744 23314 raft_consensus.cc:2889] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:29.814870 28859 consensus_meta.cc:287] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.693s	user 0.002s	sys 0.007s
I20260709 12:15:29.872557 28948 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:15:29.880806 28770 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:29.900816 28673 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:29.906708 23434 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:29.880386 28857 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.000s	sys 0.010s
I20260709 12:15:29.899569 28942 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:29.895155 28895 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.897918 28473 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:29.920073 28882 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:30.047981 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 88221.4us
I20260709 12:15:29.894146 28959 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.931957 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:29.917635 28821 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:29.933869 23463 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:29.920006 28964 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.969062 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 12:15:29.973242 28949 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:30.079867 28955 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.196s	user 0.004s	sys 0.000s
I20260709 12:15:30.082527 28753 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.084012 28922 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:29.975009 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:29.974340 28884 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:29.975764 28956 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:29.894186 28799 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.966746 28871 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:29.960381 23312 rpcz_store.cc:276] 0709 12:15:28.847085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.847233 (+   148us) service_pool.cc:224] Handling call
0709 12:15:28.847592 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:28.852214 (+  4622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:28.852242 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:28.858974 (+  6732us) log.cc:844] Serialized 26 byte log entry
0709 12:15:29.617296 (+758322us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:29.617340 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.617410 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:29.898100 (+280690us) raft_consensus.cc:1685] finished
0709 12:15:29.898132 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:29.898330 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1872,"raft.queue_time_us":7321,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4016,"thread_start_us":697635,"threads_started":1,"wal-append.queue_time_us":723139}
W20260709 12:15:29.980425 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 81748.5us
W20260709 12:15:29.974404 28874 consensus_meta.cc:287] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.001s	user 0.010s	sys 0.002s
I20260709 12:15:29.934293 28926 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:30.046342 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 139757us
I20260709 12:15:29.981508 28938 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:30.050176 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.099642 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:30.099911 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:29.981885 28947 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:15:30.106775 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.058205 28964 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.107736 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.108347 23313 raft_consensus.cc:2410] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:30.064929 28877 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.022281 28958 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:29.989596 28859 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.022418 28770 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:29.983896 28788 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:30.092870 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:30.094580 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 440642us
I20260709 12:15:29.977512 28742 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.178447 28742 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:30.099054 23206 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 130145us
I20260709 12:15:30.181715 23469 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 9. Current term is 10. Ops: []
W20260709 12:15:30.101925 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.052225 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:30.051811 28944 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:30.022544 28857 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.000s	sys 0.010s
I20260709 12:15:30.110041 28821 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:30.150669 28958 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.109144 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.111835 28648 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:30.193341 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1445) took 1187 ms. Trace:
W20260709 12:15:30.112637 28792 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.004s
I20260709 12:15:29.981333 28876 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.111442 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:30.112850 28042 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:30.114034 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2553) took 1184 ms. Trace:
W20260709 12:15:30.231480 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120107us
I20260709 12:15:30.114964 28848 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.115324 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2556) took 1086 ms. Trace:
I20260709 12:15:30.231619 23462 rpcz_store.cc:276] 0709 12:15:28.929938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.971621 (+ 41683us) service_pool.cc:224] Handling call
0709 12:15:30.113329 (+1141708us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b58000c3c98. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654feee 000000000036eb39 00007ff0ae5c31e0 00007ff0ae68c0c4 00007ff0ae68c131 00007ff0ae64dfc0 00007ff0ae638929 00007ff0ae6383d2 00007ff0a7bafbd4
0709 12:15:30.113526 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.113716 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.113743 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.113789 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:30.113827 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.113879 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.113979 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7018624}
I20260709 12:15:30.236335 23463 rpcz_store.cc:276] 0709 12:15:29.028786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.029569 (+   783us) service_pool.cc:224] Handling call
0709 12:15:29.029836 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.055887 (+1026051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.115067 (+ 59180us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54002d96d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.115257 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071539968}
I20260709 12:15:30.119735 28787 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.120751 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 2721) took 1203 ms. Trace:
I20260709 12:15:30.239147 23321 rpcz_store.cc:276] 0709 12:15:28.917529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.933485 (+ 15956us) service_pool.cc:224] Handling call
0709 12:15:30.120476 (+1186991us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b5400388454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:30.120668 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578206592}
I20260709 12:15:30.122552 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2558) took 1022 ms. Trace:
I20260709 12:15:30.240247 23470 rpcz_store.cc:276] 0709 12:15:29.099840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.100245 (+   405us) service_pool.cc:224] Handling call
0709 12:15:29.100659 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:29.101065 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.101106 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:29.133712 (+ 32606us) log.cc:844] Serialized 26 byte log entry
0709 12:15:29.596975 (+463263us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:29.597020 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.597099 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:30.122350 (+525251us) raft_consensus.cc:1685] finished
0709 12:15:30.122373 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.122497 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7366,"raft.queue_time_us":208720,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":9588,"thread_start_us":378815,"threads_started":1,"wal-append.queue_time_us":540780}
I20260709 12:15:30.123219 28882 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.124279 28959 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.130527 28790 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:30.133092 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.110208 28953 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.252540 28953 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.253619 28673 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.254138 28673 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.139748 23465 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:30.260535 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2719) took 1364 ms. Trace:
I20260709 12:15:30.261154 23317 rpcz_store.cc:276] 0709 12:15:28.896217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.092272 (+196055us) service_pool.cc:224] Handling call
0709 12:15:29.138207 (+ 45935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:29.315828 (+177621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.260213 (+944385us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54001a7cd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.260452 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339189504}
I20260709 12:15:30.262844 28882 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.262748 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:30.263592 23317 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:30.143355 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:30.141903 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.264384 28958 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:30.264776 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.265193 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.267716 28876 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:30.267890 23313 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1435) took 1430 ms (client timeout 1600 ms). Trace:
I20260709 12:15:30.268375 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:30.268329 23313 rpcz_store.cc:269] 0709 12:15:28.837205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.838033 (+   828us) service_pool.cc:224] Handling call
0709 12:15:30.267722 (+1429689us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540001cf54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:30.267839 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2919177600}
I20260709 12:15:30.269985 23394 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:29.978809 28963 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.144302 28880 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:30.271171 28880 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.272414 28963 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.100111 28926 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:30.147603 23316 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:30.273586 28635 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.274276 23387 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:30.274811 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.275689 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:30.148792 28964 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:30.149329 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.109633 28938 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.276453 23390 raft_consensus.cc:2376] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12. Re-sending same reply.
I20260709 12:15:30.237879 28857 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:30.277029 24015 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:30.277971 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.278172 28938 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:30.154079 28967 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:15:30.278479 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.156774 28884 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:30.279269 23318 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:30.280423 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.146682 24013 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:30.159403 28927 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:30.281635 28927 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:30.283429 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2709) took 1296 ms. Trace:
I20260709 12:15:30.174531 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1254) took 1273 ms. Trace:
I20260709 12:15:30.283949 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.284449 24016 rpcz_store.cc:276] 0709 12:15:28.900703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.900820 (+   117us) service_pool.cc:224] Handling call
0709 12:15:30.174253 (+1273433us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b540011d5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:30.174450 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180187520}
I20260709 12:15:30.176028 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1429) took 1340 ms. Trace:
I20260709 12:15:30.285027 23313 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:30.175882 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83144.2us
I20260709 12:15:30.177709 28764 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:30.182932 24012 raft_consensus.cc:606] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:30.180699 28788 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:30.111929 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.194854 23315 rpcz_store.cc:276] 0709 12:15:29.006596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.795892 (+789296us) service_pool.cc:224] Handling call
0709 12:15:29.820493 (+ 24601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:29.890032 (+ 69539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.890090 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:29.890749 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 12:15:29.951580 (+ 60831us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:29.951645 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.079349 (+127704us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b54003af554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.079420 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:30.161912 (+ 82492us) raft_consensus.cc:1685] finished
0709 12:15:30.161939 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.193248 (+ 31309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":329,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"spinlock_wait_cycles":158755072,"thread_start_us":60258,"threads_started":1,"wal-append.queue_time_us":209026}
I20260709 12:15:30.202024 23319 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:30.230163 23389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2698) took 1981 ms (client timeout 1620 ms). Trace:
I20260709 12:15:30.250294 28473 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:30.263509 28882 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:30.274077 28635 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.279066 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:30.256451 28673 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.176523 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:30.302232 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.286286 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.287633 23395 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:15:30.283828 23396 rpcz_store.cc:276] 0709 12:15:28.987072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.008626 (+ 21554us) service_pool.cc:224] Handling call
0709 12:15:29.009180 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:29.226868 (+217688us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b4800002dd8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:15:29.468380 (+241512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.468529 (+   149us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:29.502633 (+ 34104us) log.cc:844] Serialized 26 byte log entry
0709 12:15:29.880646 (+378013us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:29.880687 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:29.880840 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:30.283121 (+402281us) raft_consensus.cc:1685] finished
0709 12:15:30.283144 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.283371 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23890,"raft.queue_time_us":111199,"raft.run_cpu_time_us":2110,"raft.run_wall_time_us":94858,"spinlock_wait_cycles":8272640,"thread_start_us":657907,"threads_started":2,"wal-append.queue_time_us":325281}
I20260709 12:15:30.289044 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.311918 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.285271 23320 rpcz_store.cc:276] 0709 12:15:28.835658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.839029 (+  3371us) service_pool.cc:224] Handling call
0709 12:15:30.174271 (+1335242us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400148f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:30.175970 (+  1699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360341376}
I20260709 12:15:30.290681 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2728) took 1129 ms. Trace:
I20260709 12:15:30.276121 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.293192 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:30.291626 23389 rpcz_store.cc:269] 0709 12:15:28.217953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.598522 (+380569us) service_pool.cc:224] Handling call
0709 12:15:30.198959 (+1600437us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b540043c1d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:30.199116 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173646208}
I20260709 12:15:30.297619 23321 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:30.313617 23314 rpcz_store.cc:276] 0709 12:15:29.161210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.842889 (+681679us) service_pool.cc:224] Handling call
0709 12:15:29.843294 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:29.979350 (+136056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:29.979382 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:29.981133 (+  1751us) log.cc:844] Serialized 26 byte log entry
0709 12:15:30.097878 (+116745us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:30.097916 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.259742 (+161826us) spinlock_profiling.cc:243] Waited 33.6 ms on lock 0x7b54003f98d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.259814 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:30.283245 (+ 23431us) raft_consensus.cc:1685] finished
0709 12:15:30.283263 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:30.290597 (+  7334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77371776,"thread_start_us":116469,"threads_started":1,"wal-append.queue_time_us":204469}
I20260709 12:15:30.026856 28894 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:30.295236 28875 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.654s	user 0.006s	sys 0.005s
I20260709 12:15:30.318099 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.298540 23465 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:15:30.299607 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.304827 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.335510 23462 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:30.304956 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 128497us
I20260709 12:15:30.336691 23313 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:15:30.337662 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.282565 28927 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.305819 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.343981 23470 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:30.306058 23467 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:15:30.304545 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.306919 23393 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:30.302373 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:30.306739 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.294054 28042 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.304080 28764 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.329936 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.348278 23465 pending_rounds.cc:85] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 6
I20260709 12:15:30.329618 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.230594 28792 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.330246 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.331640 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:30.349083 23394 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:30.317792 28736 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:30.272472 28880 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.317370 23314 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:15:30.333088 23321 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:15:30.333856 23387 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
W20260709 12:15:30.298040 28923 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 12:15:30.305553 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.355257 23464 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:30.318235 28924 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:30.329617 28857 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.348578 23465 pending_rounds.cc:107] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:15:30.332466 23316 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:15:30.350099 23461 raft_consensus.cc:2368] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
I20260709 12:15:30.310050 28854 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.346262 28929 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:15:30.317979 28648 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:30.345090 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.369733 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.370445 23467 raft_consensus.cc:2393] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 8: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:30.334734 28894 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.290812 28546 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.356132 23391 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:30.379171 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.355309 28923 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:30.356680 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.355332 23468 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:30.380527 23466 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:30.381027 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.379878 23316 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
W20260709 12:15:30.353775 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52233.2us
I20260709 12:15:30.382619 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.383472 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:30.384027 23387 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:30.363292 23982 connection.cc:256] RPC call timeout handler was delayed by -1.3813861389999147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
I20260709 12:15:30.365272 23465 raft_consensus.cc:2889] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:30.381832 23468 raft_consensus.cc:2410] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:30.373607 24009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57b150e0c7494c95beb04642f466f450"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:30.385331 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2557) took 1341 ms. Trace:
I20260709 12:15:30.385969 24009 raft_consensus.cc:606] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.385900 23465 rpcz_store.cc:276] 0709 12:15:29.043897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.044021 (+   124us) service_pool.cc:224] Handling call
0709 12:15:29.044303 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.139195 (+1094892us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5800928ad0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4
0709 12:15:30.385115 (+245920us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.385282 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469634816}
I20260709 12:15:30.368237 28736 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.375614 28894 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.376276 28648 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.353685 28325 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:30.394411 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.395154 23467 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:30.356377 28959 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:30.396086 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.358548 28940 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:15:30.363265 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:30.373055 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:30.386484 24009 raft_consensus.cc:614] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:30.450973 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.387157 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.388336 23317 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:30.388185 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:30.390131 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.455338 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.355779 28790 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.350549 28944 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:15:30.457875 28790 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.460112 28790 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.377903 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:30.462515 23321 raft_consensus.cc:2410] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:30.367092 28875 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.463305 28875 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.370190 28792 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.465348 28875 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:30.469645 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 106437us
I20260709 12:15:30.471311 23321 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:30.366706 28964 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.355906 28842 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:15:30.389371 28546 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.353578 28042 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:30.378264 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.392798 28880 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:30.388777 28923 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.449491 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.480573 23466 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:30.378755 28854 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.392124 28732 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.374985 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.456453 23464 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:30.455979 23462 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:30.455744 23315 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:30.472327 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:30.472056 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.369637 28857 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:30.489777 28978 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:15:30.491556 28790 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:30.494751 23463 raft_consensus.cc:2410] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:15:30.479463 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:30.367704 28924 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.495707 23392 raft_consensus.cc:606] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:30.451781 28959 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:30.452863 23387 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:30.445648 28963 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.499406 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:30.363567 28788 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.484333 23461 raft_consensus.cc:2376] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12. Re-sending same reply.
I20260709 12:15:30.486616 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.445374 28859 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.478587 28875 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.515686 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:30.522791 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:30.482878 28923 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:30.530841 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.536787 23467 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
W20260709 12:15:30.539029 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 153849us
I20260709 12:15:30.489259 28546 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:30.539958 23982 leader_election.cc:336] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.604s (SENT)
I20260709 12:15:30.543395 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:30.544118 23982 connection.cc:256] RPC call timeout handler was delayed by -1.1482861389999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19710000000000003s for a response
W20260709 12:15:30.544631 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.544715 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.477795 28964 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.548154 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.549193 24012 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:30.550431 23466 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:30.550895 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.527691 28821 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.538328 28958 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:30.585520 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.512012 28792 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:30.576790 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.587325 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.502170 28732 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.590718 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.591470 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.591701 24011 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:30.479902 28764 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.575531 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.527962 28878 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:15:30.582126 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:30.560256 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.576413 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.596194 23982 leader_election.cc:400] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16.
I20260709 12:15:30.590912 24016 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:30.580448 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.549748 28848 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:30.591328 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.596534 23982 leader_election.cc:403] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:15:30.601461 24010 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:30.542546 28787 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:30.604022 23982 leader_election.cc:400] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19.
I20260709 12:15:30.604338 23982 leader_election.cc:403] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:15:30.545688 28673 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.596390 24007 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:30.495021 28736 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.609015 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.537465 28875 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.575999 28648 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:30.605473 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.512257 28963 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.576375 28949 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:30.614326 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:30.584964 28958 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:30.593421 28980 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:30.627767 24015 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:30.606799 28799 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:30.610214 28842 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16.
I20260709 12:15:30.644225 28963 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:30.678886 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.647065 28732 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.647395 28924 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.650777 28964 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.641585 28958 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:30.653164 24011 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:30.668632 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:30.698411 23315 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:15:30.669936 28981 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:15:30.575363 28959 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.608089 28792 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:30.700205 28792 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.646690 28983 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:30.709630 28983 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:30.710016 28983 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.710309 28875 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.615945 28673 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.710925 28983 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.712152 28983 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:30.712795 28673 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.615425 28888 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19.
I20260709 12:15:30.690416 28799 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.714239 28799 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.716480 28926 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.718238 28949 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:30.719127 23314 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
W20260709 12:15:30.684252 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66913.6us
I20260709 12:15:30.722798 28926 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:30.587584 28821 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.634219 28882 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:30.611856 28972 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:15:30.649997 28042 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.619971 28884 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:15:30.728698 24013 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:30.733098 24014 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:30.723517 28848 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:30.744537 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 131955us
I20260709 12:15:30.743011 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.750308 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.739586 28878 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:15:30.769291 28870 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:30.772488 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.775287 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:30.776449 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.776889 28821 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.777653 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.777825 28997 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:30.778384 28927 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:30.778913 28997 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:30.779270 28997 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:30.779663 28845 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:30.780015 28997 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.740712 28888 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:15:30.784360 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.785074 23463 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:30.781196 28997 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:30.788548 23321 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 9. Current term is 10. Ops: []
I20260709 12:15:30.788867 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.780030 28845 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:30.789860 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.821031 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:30.791023 28933 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.792007 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.792778 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.771662 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.793840 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.788909 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.790887 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.786854 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:30.795753 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.796568 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.796850 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.797726 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2720) took 1890 ms. Trace:
I20260709 12:15:30.798950 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1145) took 1966 ms. Trace:
I20260709 12:15:30.802888 28958 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.786852 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:30.806105 28940 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:30.806107 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.823140 24010 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:30.823575 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:30.833734 23461 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:30.833418 23464 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:30.835064 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.835814 23319 rpcz_store.cc:276] 0709 12:15:28.906989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:29.723188 (+816199us) service_pool.cc:224] Handling call
0709 12:15:29.752121 (+ 28933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.291439 (+539318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.797391 (+505952us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b5400149454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.797641 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034134400}
I20260709 12:15:30.808578 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.837276 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:30.837374 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50590.4us
I20260709 12:15:30.848410 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.849804 23463 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
W20260709 12:15:30.838999 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51740.9us
I20260709 12:15:30.833235 23469 rpcz_store.cc:276] 0709 12:15:28.832671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.871836 (+ 39165us) service_pool.cc:224] Handling call
0709 12:15:28.872130 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.182174 (+1310044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:30.798629 (+616455us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003dac54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:30.798856 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3011285888}
I20260709 12:15:30.854544 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.856029 23318 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:15:30.856045 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.856974 23468 raft_consensus.cc:2410] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:15:30.857868 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:30.858369 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:30.859930 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:30.845794 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:30.822119 28982 raft_consensus.cc:993] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:30.862351 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.871948 23461 raft_consensus.cc:2368] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
I20260709 12:15:30.873255 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.874248 23461 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:30.875002 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.875937 23461 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:30.875957 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:30.876726 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.862923 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.852866 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.863143 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.880015 23468 raft_consensus.cc:2376] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10. Re-sending same reply.
I20260709 12:15:30.862090 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.865010 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.880537 28959 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:30.881068 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.861140 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.881472 23464 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:30.877280 23461 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:30.852602 28821 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:30.900843 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.922427 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.838915 28942 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:15:30.824247 28546 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
W20260709 12:15:30.922314 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61360.2us
I20260709 12:15:30.839085 28968 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:30.924755 28949 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:30.883706 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.944761 28968 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.945230 23463 raft_consensus.cc:2376] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12. Re-sending same reply.
I20260709 12:15:30.924793 28845 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:30.935751 28958 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:30.935649 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:30.949947 23461 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:30.898144 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:30.862054 28878 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:15:30.881115 28959 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:30.848507 28842 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:30.871138 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 193951us
I20260709 12:15:30.877076 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.876955 28926 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:30.924760 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:30.943215 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:30.989662 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46582.6us
W20260709 12:15:30.990106 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 113118us
I20260709 12:15:30.990818 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.848235 28972 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:30.898144 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.918937 28982 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:30.948697 23463 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:30.949734 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 86850.6us
I20260709 12:15:30.877744 28744 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:30.936210 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.002562 23464 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:31.003402 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.004225 23464 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:30.928417 28929 raft_consensus.cc:1081] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:30.869952 28870 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:30.870712 28933 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:30.995807 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:30.944867 28949 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:30.945402 28968 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:30.998165 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.940847 28923 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:31.001152 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 171536us
I20260709 12:15:31.010002 24011 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:31.000244 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.012916 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:30.989387 28926 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:31.066419 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 57102.4us
W20260709 12:15:31.066654 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53845.7us
I20260709 12:15:30.996953 28972 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.998140 28042 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.071038 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.098403 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:31.061823 23462 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:31.102144 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.009071 28744 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:30.996888 28821 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:31.108197 28042 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.069059 28940 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:31.112140 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.113508 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.113559 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.020043 28949 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:31.114975 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:31.011771 28870 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.068765 28648 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:31.109622 28964 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:31.103642 28792 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:31.008463 28924 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:31.013157 28845 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.120770 23287 leader_election.cc:400] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13.
I20260709 12:15:31.115366 28744 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:31.130340 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.131412 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:31.119630 29003 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 12:15:31.102430 28972 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.081069 28859 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:31.124397 23463 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.066027 28926 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:31.125545 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58863.7us
I20260709 12:15:31.044490 28875 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.111785 28929 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.132740 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.138706 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.138633 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.138388 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.153965 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:31.070793 28937 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:15:31.137946 28648 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.148218 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.148880 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:31.150228 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 150061us
I20260709 12:15:31.150756 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.151408 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.153818 23389 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:31.125511 28878 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:31.155159 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.162609 23388 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:31.126621 28870 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.158932 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:31.160070 23392 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.138361 28970 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:31.135629 28963 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:31.161294 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 374472us
I20260709 12:15:31.156447 28929 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.193408 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:31.161885 23396 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:31.164489 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.155913 28932 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.195075 23390 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:31.152783 28884 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:15:31.195780 28932 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.196341 28884 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:31.196738 28884 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:31.167085 28964 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:31.197852 28964 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.197626 28884 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.198246 28964 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.199560 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.200188 23389 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:31.201730 24013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:31.198246 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.202601 28959 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:31.177277 29000 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:15:31.203003 23393 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:31.203526 23387 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:31.186399 28821 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:31.152693 28982 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.204789 28982 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.219210 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.195838 28546 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:31.223628 28546 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:31.224611 28546 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.225014 28854 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.166174 28937 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:31.157510 28875 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.226234 28937 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:31.197171 28970 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:31.202072 28884 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:31.227113 28937 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.227311 28970 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:31.186019 29011 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:31.234694 28876 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.235390 28876 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.238492 23393 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:31.259217 28880 raft_consensus.cc:993] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:15:31.259876 28880 raft_consensus.cc:1081] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:15:31.262465 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.157960 28859 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:31.326545 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 161648us
W20260709 12:15:31.326434 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 133099us
I20260709 12:15:31.328742 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.330345 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:31.332647 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.338531 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.332792 28854 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:31.345046 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.330054 28937 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:31.350265 24015 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:31.347712 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:31.342613 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.352191 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:31.352912 23463 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:31.333920 28923 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.341274 28821 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:31.339366 28648 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:31.367873 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.360080 23392 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:31.353741 28857 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.387495 28876 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.388384 29017 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.388898 29017 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.389282 29017 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.398383 28859 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.401759 28042 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:31.405718 28648 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.407176 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.413579 28982 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.414214 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:31.414654 29013 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:15:31.415712 28940 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:31.417002 28940 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:31.418757 28940 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:31.419209 28940 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:31.419663 28940 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:31.376909 28923 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:15:31.422586 29009 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:15:31.423131 29009 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:31.424386 28875 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.424194 29009 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.428063 28635 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.429301 28732 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:31.430267 28732 consensus_queue.cc:1243] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:15:31.430646 28983 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.718s	user 0.016s	sys 0.004s
I20260709 12:15:31.430969 28938 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:31.431419 28938 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:31.432353 28997 consensus_meta.cc:287] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.612s	user 0.012s	sys 0.004s
I20260709 12:15:31.430292 28770 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.384544 28929 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.432479 28938 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.437094 28938 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:31.437879 29009 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:15:31.450650 28927 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.672s	user 0.014s	sys 0.003s
I20260709 12:15:31.460254 28792 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.461674 28648 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:31.462241 28648 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.463536 28953 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.464275 28648 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:31.464768 28648 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.466562 23462 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:31.468091 23321 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:31.469203 23321 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:31.460847 28792 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.470952 23462 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:31.476826 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:15:31.470132 23463 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:31.477351 23316 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:31.477217 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:31.477857 23316 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:31.478129 23315 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.476315 28842 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.481312 23314 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:15:31.481585 28998 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:31.481918 23314 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:31.482326 23314 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.482621 28325 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:31.483023 28325 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:31.483855 28325 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.485025 28325 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:31.485832 28792 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.480872 28842 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.487762 28842 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.465152 28648 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:31.489012 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.498790 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.499447 23463 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:31.501808 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2719) took 1190 ms. Trace:
I20260709 12:15:31.503866 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1460) took 1130 ms. Trace:
I20260709 12:15:31.504359 23312 rpcz_store.cc:276] 0709 12:15:30.373829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.390088 (+ 16259us) service_pool.cc:224] Handling call
0709 12:15:30.390373 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.467180 (+ 76807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.467212 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:30.467258 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:30.467294 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.503565 (+1036271us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b5400389854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:31.503629 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.503792 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176188800}
I20260709 12:15:31.497021 23315 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:31.507789 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.508289 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.510514 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.505254 23314 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.502285 23391 rpcz_store.cc:276] 0709 12:15:30.311542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.311689 (+   147us) service_pool.cc:224] Handling call
0709 12:15:31.501595 (+1189906us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:31.501732 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100268800}
I20260709 12:15:31.509114 23315 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:31.515002 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.515736 23315 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:31.453835 28857 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.537184 23314 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.542482 23316 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.543807 23312 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:31.500787 28854 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:31.553259 23316 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:31.547744 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 415051us
I20260709 12:15:31.498942 29015 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:31.532932 28742 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:31.566509 29017 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.518658 28732 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 12:15:31.532651 28842 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:31.581816 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:31.583081 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.583968 23312 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:31.557225 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.509701 28998 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:15:31.589123 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:31.602576 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 234779us
I20260709 12:15:31.574939 28742 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.606669 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:31.592859 23466 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.003s	sys 0.014s
I20260709 12:15:31.602777 23314 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:31.607460 24014 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:31.608019 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.599509 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:31.609242 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.610101 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1157) took 1255 ms. Trace:
I20260709 12:15:31.610325 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.592237 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.604111 28854 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:31.587505 28842 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.619566 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.598315 28980 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:31.621493 24014 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 20 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:15:31.623353 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.610603 23466 rpcz_store.cc:276] 0709 12:15:30.354262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.675670 (+321408us) service_pool.cc:224] Handling call
0709 12:15:30.685257 (+  9587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:30.689124 (+  3867us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:30.997671 (+308547us) log.cc:844] Serialized 14 byte log entry
0709 12:15:31.608762 (+611091us) log.cc:844] Serialized 14 byte log entry
0709 12:15:31.609033 (+   271us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:31.609081 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:31.609203 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.609272 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.609372 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120002,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":60813,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"thread_start_us":865431,"threads_started":3,"wal-append.queue_time_us":138183}
I20260709 12:15:31.632205 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:31.634299 24016 raft_consensus.cc:2376] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15. Re-sending same reply.
I20260709 12:15:31.634836 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.640740 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.641669 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.642419 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.643757 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.643944 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.644476 23389 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:31.645177 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.646636 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.607476 28929 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:31.647830 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:31.656154 23465 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.657967 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.645346 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.658762 23463 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:31.648221 23394 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
W20260709 12:15:31.621580 28933 consensus_meta.cc:287] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.601s	user 0.018s	sys 0.000s
I20260709 12:15:31.649628 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.650069 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:31.663774 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:31.664412 23467 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:31.664525 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:31.665062 23395 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:31.665357 23395 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:31.663815 24015 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:31.666409 24015 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:31.650897 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.671939 24008 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:31.648672 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:31.660804 28821 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:31.659062 23390 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:31.673511 24012 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:31.616252 28770 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.682086 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.683941 23465 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:31.667105 24015 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.687736 24015 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.688620 23467 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:31.632010 28842 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:31.697587 23393 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:31.663673 28876 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:31.663132 28959 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:31.659596 28963 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:31.682672 28972 consensus_meta.cc:287] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.006s
I20260709 12:15:31.614539 28744 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.620555 28799 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:15:31.662791 28929 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:31.702625 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 331820us
W20260709 12:15:31.693327 29013 log.cc:927] Time spent T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:31.710055 24015 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:31.682183 28042 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.696924 23395 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.732627 23395 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:31.712419 28958 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:31.713572 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 98431.8us
I20260709 12:15:31.737205 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.683252 28980 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:31.660075 28848 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:15:31.724802 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.726233 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:31.774155 28980 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:31.775087 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 48930.3us
I20260709 12:15:31.716719 28959 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:31.775871 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.776624 28959 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.777037 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:31.739152 29017 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.779881 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:31.727046 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:31.727303 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.702390 28648 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:31.716094 28968 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.635s	user 0.016s	sys 0.002s
I20260709 12:15:31.683367 28770 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.692553 28964 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.705343 29018 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.741104 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.741753 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:31.741961 29021 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:15:31.749744 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 113611us
I20260709 12:15:31.753708 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:31.759099 28821 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.762763 28876 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:31.784205 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.787140 23395 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:31.730345 28929 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:31.810390 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.830668 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:31.831908 23464 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:31.833039 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:31.735661 28799 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:15:31.814801 23362 connection.cc:256] RPC call timeout handler was delayed by -1.1206815230000302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:15:31.807083 28949 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.004s
I20260709 12:15:31.836886 23317 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:15:31.722244 28972 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.818061 28876 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:31.838276 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:31.799789 28870 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.011s
I20260709 12:15:31.811959 23319 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:31.884658 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.884943 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.885131 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.885335 23390 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:31.885643 23391 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:31.840499 23394 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:31.782157 28963 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:31.854048 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.807720 28770 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.868453 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:31.878158 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:31.900245 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:31.792472 28848 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:31.887162 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:31.887267 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:31.888550 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.837406 28648 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:31.908682 23390 raft_consensus.cc:2376] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14. Re-sending same reply.
I20260709 12:15:31.910151 28848 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.910526 28848 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.910872 28848 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.891708 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:31.899204 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:31.791697 28875 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.912142 23390 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:31.912452 28923 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.913383 23390 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:31.914716 23390 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:31.901715 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:31.916525 23464 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:31.901794 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.840715 29015 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:31.739547 28042 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.919407 23391 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:15:31.920488 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.922329 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:31.882862 29007 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:15:31.898876 29032 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:15:31.925575 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:31.940613 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.899147 28882 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.814514 28968 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.882519 29017 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.863828 29036 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 12:15:31.918238 28857 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:31.946280 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:31.783309 28940 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:31.882438 29024 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:31.950495 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:31.961395 28792 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:31.962329 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:31.962929 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:31.923727 28742 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:31.954720 23469 raft_consensus.cc:2410] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:15:31.921274 29015 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:31.947324 28854 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:15:31.942350 28937 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.587s	user 0.002s	sys 0.008s
I20260709 12:15:31.865913 28949 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:31.963742 23470 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:31.928249 28848 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:31.734737 28948 consensus_queue.cc:1175] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2778b449761a46708f2f643b1b5b5854" responder_term: 11 status { last_received { term: 10 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:31.966384 28940 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.966847 28942 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:31.974709 28968 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.945011 28923 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:31.979811 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1465) took 1390 ms. Trace:
I20260709 12:15:32.014863 28648 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.981243 28972 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.982476 28964 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:31.983201 29025 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:31.955564 28882 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:31.991631 28926 consensus_meta.cc:287] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.834s	user 0.011s	sys 0.006s
I20260709 12:15:31.992559 23321 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:32.047699 28742 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:32.003479 28932 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.807s	user 0.006s	sys 0.010s
I20260709 12:15:32.010277 28770 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.095461 28949 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.063038 28744 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.069103 28648 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.096927 28770 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:32.014171 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.098129 23395 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:32.098618 28648 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:31.942778 28933 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:31.937714 28859 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.100366 28882 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.016829 23464 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:32.038964 23462 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:31.908201 28870 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.054589 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:32.104574 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:32.105155 23464 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.105718 23320 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:32.058188 23313 rpcz_store.cc:276] 0709 12:15:30.589678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.613748 (+ 24070us) service_pool.cc:224] Handling call
0709 12:15:30.625002 (+ 11254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:30.729681 (+104679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:30.729712 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:30.741868 (+ 12156us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.406008 (+664140us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b180046fc60. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:31.704837 (+298829us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:31.704873 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.707857 (+  2984us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:31.979488 (+271631us) raft_consensus.cc:1685] finished
0709 12:15:31.979511 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:31.979743 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":536,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3695,"spinlock_wait_cycles":4552960,"thread_start_us":298644,"threads_started":1,"wal-append.queue_time_us":1095077}
I20260709 12:15:31.980370 24013 raft_consensus.cc:606] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:32.034240 28970 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:32.030939 29015 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.061131 23315 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:32.064288 23465 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:32.064574 28929 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.055871 28857 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.011s
W20260709 12:15:32.130051 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.065522 28848 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:31.990792 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:31.957599 29017 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:32.069756 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 289915us
I20260709 12:15:31.996501 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:32.166581 23318 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:32.168830 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.170886 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.055783 28942 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.086768 29009 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:32.089520 23463 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:32.089746 29010 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.091526 28894 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:32.091224 28926 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.091939 28742 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.192312 28742 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.106081 28870 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.106535 23394 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:15:32.201861 23394 pending_rounds.cc:85] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 12
I20260709 12:15:32.202121 23394 pending_rounds.cc:107] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:15:32.202427 23394 raft_consensus.cc:2889] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:32.106535 24013 raft_consensus.cc:614] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:32.108217 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.204524 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:32.200943 29049 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:15:32.093101 23466 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:15:32.070043 28938 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.627s	user 0.005s	sys 0.004s
I20260709 12:15:32.206209 24007 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:32.206830 28932 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:32.207376 28932 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:32.208242 28932 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.176268 28949 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:32.225225 28949 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.227409 28949 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.134760 23314 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:32.231853 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.147161 24012 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:32.157558 28940 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.160462 28770 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.235368 28940 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.161754 28929 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.165769 23389 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:32.236732 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:32.235507 28770 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.243443 24012 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:32.250783 23314 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:32.180701 28842 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.183631 28848 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:32.175869 28942 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.187829 29010 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.176132 28894 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:32.189810 29037 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:32.190469 28926 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.262326 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.193145 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.193986 23468 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:32.074277 29025 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.194540 24008 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:32.224484 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.267755 24010 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:32.177134 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.216634 29023 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.265122 23318 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:32.258956 28933 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.276065 23463 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:32.274070 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.117340 23464 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:32.193200 28876 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.261170 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.263098 24012 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:32.278971 24010 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:32.266232 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73193us
I20260709 12:15:32.280216 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:32.258325 28848 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.312754 23388 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:32.356299 28854 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:32.358425 28854 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.363421 28933 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.364357 28876 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.261168 29037 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.365697 29037 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:32.365986 28933 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.369268 29017 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.272853 29023 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.372198 29023 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:32.374603 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 94506.6us
I20260709 12:15:32.387004 23465 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 12:15:32.420044 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44606.8us
I20260709 12:15:32.421446 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.269887 28870 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:32.431430 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.439174 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:32.440369 24009 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:32.442610 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.375586 28848 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:32.428934 28929 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.454943 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.456475 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:32.429378 28876 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:32.459673 28878 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:32.460104 28878 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.460484 28878 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:32.465685 28953 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:32.432552 28673 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.434263 28870 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.498648 28870 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.437996 28949 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:32.501647 28949 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.502111 28949 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:32.437496 28894 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.449913 29043 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:32.514180 29043 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:32.514717 29043 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.471244 28770 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.433705 28964 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:15:32.471103 28876 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.523283 28876 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:32.535965 29051 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 12:15:32.541597 29044 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:15:32.449961 28980 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.545938 28980 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.440097 29045 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:32.549487 29045 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:32.550588 29045 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.553623 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.554682 24014 raft_consensus.cc:2393] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:32.556481 29025 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.557513 29049 raft_consensus.cc:2764] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:15:32.560426 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41504 (request call id 12) took 1371 ms. Trace:
I20260709 12:15:32.562420 28922 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.560905 24013 rpcz_store.cc:276] 0709 12:15:31.188964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.201584 (+ 12620us) service_pool.cc:224] Handling call
0709 12:15:32.560174 (+1358590us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54006bce54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a966e853 00007ff0ae63c60a 00007ff0a7bc27d4 00007ff0a7bc2762 00007ff0a7bc2691 00007ff0a7bc260c 00007ff0a7bc18c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:32.560322 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1789377792}
I20260709 12:15:32.566493 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:32.567358 23319 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:32.473066 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:32.487634 28940 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.498088 28842 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:32.571202 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98249.6us
I20260709 12:15:32.548542 28980 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.585225 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.586377 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.589766 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.589972 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.590334 24014 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.590520 24009 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.588176 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.592053 23387 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.587915 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:32.583104 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.586030 23393 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:32.594786 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:32.595367 23316 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:32.597096 23392 raft_consensus.cc:2393] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:32.597373 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:32.598587 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.602571 24014 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:32.604254 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:32.606297 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:32.620648 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.588640 28964 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:32.589174 28940 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:32.635766 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:32.596887 28859 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:32.640151 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:32.620276 28953 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:32.480088 28929 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.610626 29049 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:32.606518 28938 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.572001 28922 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.657868 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:32.659610 29057 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
W20260709 12:15:32.664577 29059 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:15:32.666637 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.667007 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.667675 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.572268 28673 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.681862 28842 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.714040 24012 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:32.671953 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2776) took 1027 ms. Trace:
I20260709 12:15:32.675376 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.676489 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.709641 23387 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:32.718667 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:32.719270 23392 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:32.676777 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:32.720265 23393 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:32.679049 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:32.677073 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:32.703322 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:32.704556 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:32.711982 24014 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:32.679600 28042 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.683923 28927 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:32.727005 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50052.1us
I20260709 12:15:32.709748 29038 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.728678 23982 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:32.729274 28325 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:32.732970 28788 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:32.723198 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:32.683321 29049 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:32.670966 28940 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.743307 23470 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:32.686051 28875 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:32.749984 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:32.715709 23461 rpcz_store.cc:276] 0709 12:15:31.644914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.645794 (+   880us) service_pool.cc:224] Handling call
0709 12:15:31.646112 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.750359 (+104247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.750391 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.750876 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.957328 (+206452us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:32.047502 (+ 90174us) log.cc:844] Serialized 14 byte log entry
0709 12:15:32.649035 (+601533us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b1800408b20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287
0709 12:15:32.649191 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.649273 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.649319 (+    46us) raft_consensus.cc:1685] finished
0709 12:15:32.649424 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:32.671896 (+ 22472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112983,"prepare.queue_time_us":96384,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":2360,"raft.queue_time_us":120611,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":164,"spinlock_wait_cycles":15665408,"thread_start_us":225750,"threads_started":2,"wal-append.queue_time_us":334570}
I20260709 12:15:32.779597 23463 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:32.700157 28953 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:32.705896 28938 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.773545 24010 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:32.701541 28959 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:32.776713 29049 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:32.745590 28842 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.766570 28972 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:32.771348 29009 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:32.775003 28980 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:15:32.807716 28922 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.807788 28673 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:32.809053 29040 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.745s	user 0.004s	sys 0.004s
I20260709 12:15:32.809697 28953 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.781885 28875 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:32.766232 28927 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.780347 28770 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:15:32.780228 28997 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:15:32.827386 28942 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.001s
I20260709 12:15:32.837107 28942 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.822532 28970 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:32.852627 28970 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:32.859400 28927 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.865041 28938 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.869849 28958 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:32.870210 28929 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.870584 28958 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:32.816006 28972 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.852078 29010 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.857004 28940 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:32.828770 28894 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:32.874315 29037 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 12:15:32.876117 29037 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.878911 29037 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.840109 28942 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.840775 28878 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.885376 28842 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.873165 28972 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:32.891855 29043 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.894208 28870 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:32.984967 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 378523us
I20260709 12:15:32.993669 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2755) took 1518 ms. Trace:
I20260709 12:15:33.123378 23315 rpcz_store.cc:276] 0709 12:15:31.474711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.518561 (+ 43850us) service_pool.cc:224] Handling call
0709 12:15:31.518853 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:32.116024 (+597171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:32.992729 (+876705us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54002756d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:32.993327 (+   598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246659968}
I20260709 12:15:33.125629 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:32.999624 23321 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:33.001360 23389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2740) took 1351 ms (client timeout 1730 ms). Trace:
I20260709 12:15:33.005671 23391 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:32.771699 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:33.018908 23320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 2006) took 2160 ms (client timeout 2597 ms). Trace:
I20260709 12:15:33.024536 23316 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:33.038045 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 987) took 1149 ms. Trace:
W20260709 12:15:33.047974 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.051054 23462 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:33.054530 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.056355 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:32.769649 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.135663 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2579) took 1321 ms. Trace:
W20260709 12:15:33.136291 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 364552us
W20260709 12:15:33.136685 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 367016us
I20260709 12:15:33.057111 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:33.128295 23320 rpcz_store.cc:269] 0709 12:15:30.857726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.857857 (+   131us) service_pool.cc:224] Handling call
0709 12:15:33.018088 (+2160231us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400148f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:33.018536 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2651429760}
W20260709 12:15:33.138604 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81962.9us
I20260709 12:15:33.139129 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:33.139556 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:33.140534 23470 raft_consensus.cc:2410] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:15:33.140827 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:33.062876 23468 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 1159) took 2336 ms (client timeout 2625 ms). Trace:
I20260709 12:15:33.067044 23463 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
W20260709 12:15:33.070576 23469 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2763) took 2289 ms (client timeout 2411 ms). Trace:
I20260709 12:15:33.070210 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:33.073266 24015 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:33.081723 28042 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.081950 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.082708 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1171) took 1361 ms. Trace:
I20260709 12:15:32.906133 28964 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:32.898172 28878 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:32.898527 28876 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:32.891759 28949 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.175593 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:32.786274 29038 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:32.931277 29043 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.177606 28859 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:33.144359 23467 rpcz_store.cc:276] 0709 12:15:31.721416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.721657 (+   241us) service_pool.cc:224] Handling call
0709 12:15:31.722141 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:31.752483 (+ 30342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:31.752515 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:31.752923 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 12:15:31.957771 (+204848us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:31.957816 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:31.957883 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:32.808386 (+850503us) spinlock_profiling.cc:243] Waited 5 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:33.046096 (+237710us) raft_consensus.cc:1685] finished
0709 12:15:33.046139 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:33.082473 (+ 36334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29903,"raft.queue_time_us":31619,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":11498880,"thread_start_us":202878,"threads_started":1,"wal-append.queue_time_us":300919}
W20260709 12:15:33.099186 28982 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.016s	sys 0.030s
I20260709 12:15:33.178391 28842 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:15:33.178907 28982 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.112s	user 0.016s	sys 0.031s
I20260709 12:15:33.178655 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.179299 24010 raft_consensus.cc:606] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:33.179816 24010 raft_consensus.cc:614] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:33.180847 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.181245 28042 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.182266 29010 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.183635 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:33.184108 28042 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.185318 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1265) took 1067 ms. Trace:
I20260709 12:15:33.185966 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.186115 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.186554 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.185765 24015 rpcz_store.cc:276] 0709 12:15:32.117929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.118312 (+   383us) service_pool.cc:224] Handling call
0709 12:15:32.118603 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:33.072814 (+954211us) spinlock_profiling.cc:243] Waited 82.8 ms on lock 0x7b54004d5ba0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:15:33.184812 (+111998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.185198 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190374144}
I20260709 12:15:33.188763 23391 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:33.189850 23318 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:33.192239 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.192371 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.193029 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:33.188133 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44744.5us
I20260709 12:15:33.194128 23392 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 10.
I20260709 12:15:33.195122 28842 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:33.194730 28964 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.196411 28842 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.109273 28972 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.211556 28972 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.109778 23393 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:33.115245 23395 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:33.117583 29037 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.121624 29068 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:33.215314 29037 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:33.095739 29006 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.007s
W20260709 12:15:33.216657 29006 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.004s	sys 0.007s
W20260709 12:15:33.127378 23389 rpcz_store.cc:269] 0709 12:15:31.649242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.649405 (+   163us) service_pool.cc:224] Handling call
0709 12:15:33.000764 (+1351359us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:33.001010 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1154947840}
W20260709 12:15:33.134068 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.217993 28673 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:33.218520 28673 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.219916 28673 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.221527 28972 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.141839 23321 raft_consensus.cc:2368] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:33.223387 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:32.897063 28848 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:33.141656 23468 rpcz_store.cc:269] 0709 12:15:30.726902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.922143 (+195241us) service_pool.cc:224] Handling call
0709 12:15:33.062258 (+2140115us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400186ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:33.062601 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887858560}
W20260709 12:15:33.142259 23469 rpcz_store.cc:269] 0709 12:15:30.781270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:30.898410 (+117140us) service_pool.cc:224] Handling call
0709 12:15:33.069992 (+2171582us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54001888d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:33.070347 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219802496}
I20260709 12:15:33.136231 23462 rpcz_store.cc:276] 0709 12:15:31.813855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:32.104937 (+291082us) service_pool.cc:224] Handling call
0709 12:15:32.105216 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:33.129218 (+1024002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.135242 (+  6024us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b54003edfd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:33.135568 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2165280128}
I20260709 12:15:33.128479 23394 rpcz_store.cc:276] 0709 12:15:31.888968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:31.899138 (+ 10170us) service_pool.cc:224] Handling call
0709 12:15:31.939292 (+ 40154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:32.203404 (+264112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:33.037486 (+834082us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400345dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:33.037850 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383824768}
I20260709 12:15:33.145278 28792 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:33.227703 23314 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:33.227839 28848 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.167416 28940 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:33.172514 28932 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:33.230907 28932 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:33.141407 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:33.233366 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 92076.6us
I20260709 12:15:33.233443 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.233865 28770 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:33.234287 23314 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:33.234346 28770 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.235374 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.235734 28770 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.225111 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.174247 28876 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.175781 23319 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:32.902377 29015 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.000s
I20260709 12:15:33.101570 28942 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.104918 24009 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.513s	user 0.002s	sys 0.016s
I20260709 12:15:33.197901 23320 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:33.197893 28325 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:33.199366 28880 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:33.178658 29038 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.203306 28870 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.267549 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:33.268186 24009 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:33.174296 28875 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.197091 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.177093 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:33.209877 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.213943 28970 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.215360 29068 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.126276 28854 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:33.217665 29010 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:33.193300 23318 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:33.228711 29037 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:33.273612 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 63979.4us
I20260709 12:15:33.167080 29049 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.240618 23469 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:33.241873 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:33.245201 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52315.6us
I20260709 12:15:33.240865 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.237274 23468 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:15:33.266407 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58002.3us
I20260709 12:15:33.268447 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:33.177155 28949 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.237775 23314 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:33.243022 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:33.239394 29071 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:15:33.237188 28770 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.221302 28673 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.279537 23316 raft_consensus.cc:2393] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:33.238674 28042 raft_consensus.cc:2764] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:15:33.285748 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:33.288110 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.293064 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:33.296161 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.296249 29068 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.297238 28635 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.297348 28875 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.297850 23391 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:33.305780 23319 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:33.287671 28870 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:33.248198 28958 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.320609 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.321813 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.324054 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.329676 23312 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:33.331271 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:33.297967 28635 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.335218 23463 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.304126 28876 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:33.302380 28942 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.335752 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:33.336766 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.340629 23320 raft_consensus.cc:2410] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:33.344143 28923 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:33.345393 28923 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:33.345908 29010 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.347064 28870 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.348047 23314 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:15:33.348729 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:33.350284 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:33.350564 23465 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:33.351562 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:33.351714 28859 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:33.352111 28859 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.352459 28859 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:33.352913 28792 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:33.352809 28854 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:15:33.353387 28792 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.353842 28792 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.354494 28854 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:33.354899 28854 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.355240 28854 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.346657 29010 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.348356 28870 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.341710 23316 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:33.330951 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.342386 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.345818 28923 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.359618 29010 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:33.365903 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47627.5us
I20260709 12:15:33.359460 28876 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.340740 28042 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.362604 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.359897 28635 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:33.373476 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 71476.3us
I20260709 12:15:33.381690 23360 leader_election.cc:400] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:33.382208 23360 leader_election.cc:403] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:15:33.376667 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 179761us
I20260709 12:15:33.383680 23317 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:33.405282 28325 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:33.416606 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:15:33.411964 28982 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:33.405038 28042 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.424896 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.431329 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.419034 28635 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.433609 28929 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.433687 28635 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.434309 28929 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.436601 28635 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.436683 28929 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.429877 28325 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:33.438015 28876 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.438087 28325 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.441557 29056 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.442414 29056 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.443130 28042 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.444257 28942 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.444641 29056 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.385952 29015 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.448882 29025 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.449775 23470 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:33.449580 29025 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.422050 28970 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.455873 28923 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.457383 28923 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:33.421079 29010 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:33.462106 23463 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
W20260709 12:15:33.465858 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80783.8us
I20260709 12:15:33.449800 28958 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.478195 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.462050 29049 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.481127 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:33.483861 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98245.3us
I20260709 12:15:33.468170 29038 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:33.473803 28932 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:33.486150 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.488436 23319 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:33.488359 28997 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:15:33.490348 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.492327 23391 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:33.490590 28932 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.496171 28932 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:33.504001 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.505559 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:33.508349 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:33.510339 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.510967 23388 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:33.511037 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:33.512418 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:33.513667 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:33.513818 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.514096 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.514714 23395 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.515136 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.515738 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:33.517894 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.518748 23393 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:33.477797 28949 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:33.518028 28894 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.643s	user 0.005s	sys 0.013s
W20260709 12:15:33.525808 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:33.525352 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55017.2us
I20260709 12:15:33.526846 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.527134 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.527494 23392 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.527887 23396 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.525887 29038 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:33.531728 28949 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.534123 28949 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.535511 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.536175 29010 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:33.536819 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.529662 28842 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:33.538878 28842 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.539359 28842 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:33.543148 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.542196 28929 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:33.565629 28929 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.565649 28942 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:15:33.544023 28949 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:15:33.545287 29015 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.546458 29056 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.529325 29038 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.584450 23462 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.585352 28942 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:33.585805 28942 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.552989 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.585021 23219 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 15, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 15 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.587117 23389 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:33.524894 29025 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.588874 29056 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.589532 28942 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.590847 28942 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.592160 29056 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.593103 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:33.597636 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 67240.3us
W20260709 12:15:33.603096 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60050.2us
I20260709 12:15:33.603298 23391 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:33.552589 28859 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.556797 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:33.556962 23313 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:33.557458 28997 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:33.572407 23395 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:33.574731 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.575785 23396 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:33.482367 28792 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.612066 28964 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.612880 28964 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.613744 28792 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.581607 23392 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
W20260709 12:15:33.614593 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45386.7us
I20260709 12:15:33.606734 23388 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:33.567342 28929 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:33.619078 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:33.618377 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47441.9us
I20260709 12:15:33.624387 23462 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:15:33.626910 23219 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:33.633621 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76912.9us
I20260709 12:15:33.629073 23219 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.637694 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:33.642267 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.629675 28953 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:33.633560 29010 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:33.652555 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:33.652880 29081 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:15:33.653273 24010 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:33.654419 28982 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:33.658555 28929 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:33.659351 28958 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:33.659754 28859 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.650329 28953 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.663367 28953 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.665081 23394 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:33.665578 23462 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:33.656839 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:33.648233 28964 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.693812 28958 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:33.694252 28958 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.710696 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.692962 28842 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.710932 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:33.709926 28859 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:33.677080 28932 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.702188 28875 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:33.714756 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 77161.7us
W20260709 12:15:33.717767 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 61157.6us
I20260709 12:15:33.718787 29043 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.719600 28958 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.722615 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:33.703926 28870 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:33.713994 29056 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.722795 23219 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 14, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:33.730684 23387 raft_consensus.cc:2368] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:33.730981 23462 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
W20260709 12:15:33.744879 23982 leader_election.cc:336] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.873s (SENT)
I20260709 12:15:33.745533 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:33.748332 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107206us
I20260709 12:15:33.750278 23470 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:33.750629 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.751536 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.751863 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.752202 23465 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:33.752959 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.751516 28870 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.753575 28997 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.754243 28997 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.754634 28854 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.755416 28870 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.756346 28997 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.758497 28770 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:33.758608 28932 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:33.758661 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.759439 28770 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:33.759675 28932 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:33.759543 29056 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.760460 28932 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:33.760831 28932 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.761788 28932 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.731210 28875 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.763162 28932 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:33.764449 28875 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.762133 29056 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.765326 28958 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.751784 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.746665 29043 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.771461 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.771936 29043 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:33.772264 28964 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:33.772580 29043 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:33.772719 28964 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:33.773162 28964 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.773908 28972 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:33.774930 28923 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:33.774750 23219 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 16, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 16 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.775668 28997 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:33.776131 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.776840 23468 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:33.774751 28972 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:33.777884 28972 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:33.778309 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.778976 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.778821 28972 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.780378 28972 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:33.780503 28923 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:33.781046 28923 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.746786 29015 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:33.781968 28923 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.783237 28923 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:33.783380 29086 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:15:33.784181 28964 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.785755 28842 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.726925 29038 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:33.787143 23463 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:33.778244 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.787465 23469 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:33.757035 28854 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:33.753647 23464 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:33.785779 23219 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 11 to 12, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:33.790661 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.792493 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.794507 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.795107 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:33.827030 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.827169 23468 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:33.796656 24010 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:33.796060 23980 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.796128 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:33.821553 23467 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:33.788340 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:33.830413 23467 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 12:15:33.822249 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 111413us
I20260709 12:15:33.829738 23465 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:33.786685 29043 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.831612 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.836067 23315 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:33.831065 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.837812 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:33.838276 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45864.5us
I20260709 12:15:33.838676 23388 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:33.837813 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:33.839255 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46832.6us
I20260709 12:15:33.838953 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.834450 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.799293 28970 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:33.836252 24015 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:33.836882 23387 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:33.842053 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:33.796180 29015 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:33.842993 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:33.848526 23316 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:33.852137 23394 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:33.850838 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.820495 28673 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:15:33.853926 28953 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:33.865866 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.872052 23287 leader_election.cc:400] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:33.867547 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:33.877313 23287 leader_election.cc:403] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:33.844544 28842 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:15:33.858839 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:33.893026 24013 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:33.883044 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.855108 28792 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:15:33.887877 23316 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:33.872265 28875 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:33.897035 24013 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:33.871457 28673 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:33.870792 28953 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:33.890563 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 145713us
I20260709 12:15:33.870119 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:33.849972 28854 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:33.899813 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.900460 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.899262 23390 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:33.928565 23394 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:33.827813 29038 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.929844 23395 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:33.928745 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:33.952080 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80193.1us
I20260709 12:15:33.930919 23392 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:33.932081 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:33.854233 28970 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.953529 23387 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:33.932289 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:33.953814 23219 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:33.956576 24010 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:33.871474 29043 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.855839 28325 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:33.933101 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:33.866010 28940 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:15:33.967011 23219 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:33.982257 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:33.866246 29009 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:15:33.952261 28972 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:33.989966 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:33.981182 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:33.994880 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 127428us
I20260709 12:15:33.929486 28842 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:33.955499 28875 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:33.960963 29037 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:34.000015 23395 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:33.900941 28673 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:33.865839 28922 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:15:33.954592 28042 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:34.002254 28042 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:34.002673 28042 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.003041 28042 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:34.005924 28953 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.006032 28673 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.006910 23313 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:34.007506 28959 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:34.008060 28959 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:34.008399 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.008972 28959 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.013852 28970 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:33.900530 29015 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:33.988005 29025 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:33.995991 28972 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.000146 28770 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:34.033115 23387 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:34.082633 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:34.054728 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 60749.5us
I20260709 12:15:34.074856 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:34.075457 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44824.8us
I20260709 12:15:34.063975 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.095323 23463 raft_consensus.cc:2368] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:34.090785 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.096359 23467 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:33.995586 28854 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.093695 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:34.099185 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.101537 23468 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:34.101737 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:34.113559 23466 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:34.032825 28997 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:15:34.118786 23469 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:15:34.119061 29091 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:15:34.119071 28997 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:34.119673 28997 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:34.120056 23218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 329) took 11323 ms (client timeout 15000 ms). Trace:
I20260709 12:15:34.120599 28997 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.114360 28673 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:34.120553 23218 rpcz_store.cc:269] 0709 12:15:22.797143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:22.803204 (+  6061us) service_pool.cc:224] Handling call
0709 12:15:34.119842 (+11316638us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:29.785767 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:29.786372 (+   605us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:29.786409 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:15:29.786432 (+    23us) tablet.cc:662] Decoding operations
0709 12:15:29.881373 (+ 94941us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:29.881466 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:29.881492 (+    26us) tablet.cc:685] Acquiring locks for 26 operations
0709 12:15:30.022606 (+141114us) tablet.cc:701] Row locks acquired
0709 12:15:30.045029 (+ 22423us) write_op.cc:260] PREPARE: finished
0709 12:15:30.089581 (+ 44552us) write_op.cc:270] Start()
0709 12:15:30.089792 (+   211us) write_op.cc:276] Timestamp: P: 1783599330056204 usec, L: 0
0709 12:15:30.089825 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:15:30.117748 (+ 27923us) log.cc:844] Serialized 10301 byte log entry
0709 12:15:30.774243 (+656495us) op_driver.cc:464] REPLICATION: finished
0709 12:15:31.136500 (+362257us) write_op.cc:301] APPLY: starting
0709 12:15:31.139510 (+  3010us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:31.751885 (+612375us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:31.751986 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:32.372040 (+620054us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:32.372553 (+   513us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0709 12:15:32.372629 (+    76us) write_op.cc:312] APPLY: finished
0709 12:15:32.373309 (+   680us) log.cc:844] Serialized 228 byte log entry
0709 12:15:32.844447 (+471138us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a656a0ac 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b
0709 12:15:33.217264 (+372817us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.218088 (+   824us) write_op.cc:454] Released schema lock
0709 12:15:33.218253 (+   165us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:33.218911 (+   658us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:33.218964 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":258080,"reported_tablets":47,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":344723,"apply.run_cpu_time_us":52453,"apply.run_wall_time_us":2093524,"cfile_cache_hit":106,"cfile_cache_hit_bytes":112911,"num_ops":26,"prepare.queue_time_us":151544,"prepare.run_cpu_time_us":66490,"prepare.run_wall_time_us":553469,"raft.queue_time_us":290632,"raft.run_cpu_time_us":38250,"raft.run_wall_time_us":239674,"replication_time_us":684254,"spinlock_wait_cycles":9465728,"thread_start_us":1456594,"threads_started":5,"wal-append.queue_time_us":994015,"wal-append.run_cpu_time_us":29471,"wal-append.run_wall_time_us":1253212}]]}
I20260709 12:15:34.121945 28997 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.126173 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.127599 28842 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:34.131206 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.132757 29010 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:34.019608 28042 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:34.134263 29093 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:15:34.134568 28042 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.134789 29025 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.136971 29025 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.138700 28042 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:34.140157 28042 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.140651 28042 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.145895 28875 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.073894 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.152935 29056 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:15:34.155454 29092 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:15:34.093016 28972 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:34.085000 28964 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.201577 28042 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:34.224910 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 317764us
I20260709 12:15:34.225812 28042 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.153414 29056 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.236757 28792 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.242660 23388 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:34.237087 29097 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:34.170537 28770 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:34.170506 28949 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:34.263870 23467 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:34.261351 29097 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.266125 29100 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:34.281826 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 231912us
I20260709 12:15:34.290654 28792 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.291872 28325 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.293089 29098 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.168022 28854 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.262483 28042 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.298480 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.353228 28042 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.260219 29056 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:34.297775 29097 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:34.298049 29025 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.363608 28042 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.372996 29015 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.351442 29098 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.377022 29098 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.381044 29096 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:34.381500 29096 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:34.382293 29096 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.382864 28854 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.383558 29096 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:34.389868 28770 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.363411 28870 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:15:34.361356 29010 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:34.337386 28325 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.396495 28972 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.349961 28964 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:15:34.398705 28325 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.356477 28859 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.399480 28859 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.401882 29106 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.402415 29106 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.404551 29106 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.405118 29038 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.405184 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.406678 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.407900 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.409291 28970 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:34.410180 28970 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:34.346451 29100 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.410545 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:34.410804 28970 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:34.411238 23317 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:15:34.412456 29100 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:34.419241 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 325650us
I20260709 12:15:34.409161 29037 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:15:34.451156 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 207470us
I20260709 12:15:34.466692 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:34.468360 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 58984.7us
I20260709 12:15:34.421276 28859 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.467561 28042 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.474964 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.483155 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:34.484760 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:34.480578 29110 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:34.480839 29098 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.490736 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:34.490857 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.465493 28922 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:34.494571 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 201325us
I20260709 12:15:34.461920 28854 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:15:34.499889 23319 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:34.489636 28940 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:34.503603 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.507375 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:34.493378 28859 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:34.508148 23394 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:34.490469 29110 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:34.420270 29025 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.514281 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.497279 29037 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:34.517323 28042 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.517421 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.517325 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.521363 23393 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:34.525732 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.526386 23392 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:34.535054 28792 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:34.536171 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45526.8us
I20260709 12:15:34.537159 28923 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:34.538321 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.539276 29118 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:34.539731 29118 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.540073 29118 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.549899 28859 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:34.551118 28859 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.556449 28042 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.503641 28940 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:34.557024 28042 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.558521 28940 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.560146 28940 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:34.559378 28042 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.487725 28997 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:15:34.563064 29010 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:34.563460 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.564738 28997 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:34.573542 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53271.7us
I20260709 12:15:34.580274 28042 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.580634 29097 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.580957 28042 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.509887 28673 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:34.582885 28673 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:34.583118 28042 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.583307 28673 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:34.585830 29118 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.593379 23317 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:34.510057 28929 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:34.598634 23287 leader_election.cc:400] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14.
I20260709 12:15:34.565207 28997 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.496996 29098 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.606293 29110 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.606977 29118 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.612927 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.630724 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.617980 29068 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.618693 29043 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.502678 28922 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.600420 29037 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:34.620352 29116 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:15:34.623243 29098 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.637673 23394 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:34.639645 29068 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.651831 29068 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.643932 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:34.604429 28997 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:34.646425 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 117675us
I20260709 12:15:34.642266 28875 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:34.642946 28972 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:34.668985 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 105740us
I20260709 12:15:34.645252 28964 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:34.690681 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:34.693694 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:34.694610 23394 raft_consensus.cc:2410] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:15:34.696475 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.697041 23467 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.697564 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:34.699903 23466 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:34.700999 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:34.707250 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.709223 23468 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:34.709107 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:34.692756 28959 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:34.709931 23469 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:34.713126 29037 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.713724 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.715761 28922 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.698426 29118 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:34.717659 29118 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.718019 29118 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:34.719163 28673 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.724283 23317 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:15:34.727699 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:34.730438 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.731314 24012 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:34.731701 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:34.732475 23317 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:34.734266 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:34.739902 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.741755 28042 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:34.742229 28042 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:34.743403 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.669623 29043 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.774097 29043 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.774209 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.743646 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:34.707700 28894 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:34.748081 23320 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:34.749193 29097 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.749410 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.749634 29098 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:34.750072 29121 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:15:34.750120 23464 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 12:15:34.750437 23390 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:34.750955 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:34.751214 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:34.794149 24013 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.751816 28042 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.753201 24009 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:34.753896 23316 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:34.756880 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:34.757831 24015 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:34.758311 28859 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.761238 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.762602 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.796546 23461 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.763608 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.764029 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.764317 29104 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:34.764762 23317 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:34.797317 23387 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:34.765707 29112 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:34.766585 29056 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.801272 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.737812 28959 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:34.770220 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:34.772044 28842 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:34.807050 23396 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:34.808521 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.779241 23467 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:34.810580 23465 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:34.780797 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:34.790052 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.797060 23394 raft_consensus.cc:2410] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:34.798915 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:34.811479 23391 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:34.811831 23318 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:15:34.781137 28894 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:34.787374 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.817482 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.818192 23389 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:34.822027 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 65214.7us
I20260709 12:15:34.822808 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.820376 23466 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:34.822729 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.801983 29112 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:34.828598 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.827066 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.831331 23391 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260709 12:15:34.820636 28325 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:34.839049 23390 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:34.839874 29097 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:34.840307 29097 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.840658 29097 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:34.799708 29098 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.819958 28673 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.842444 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.844923 24016 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:34.846933 28959 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:34.841553 28325 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.849074 28325 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.849084 24009 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:15:34.852751 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:34.853637 23387 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:34.844848 29105 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:34.860570 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.858383 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:34.862097 23389 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:34.862587 28042 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:34.862964 28042 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:34.863305 28042 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:34.810072 28859 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.865067 24012 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:34.827625 29104 consensus_queue.cc:1243] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:15:34.865980 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.867559 24013 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:34.841714 29068 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.870772 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:34.871104 29115 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:34.872192 23463 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:34.879140 23318 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:34.880378 28792 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:34.880801 28792 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:34.881168 28792 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:34.881610 29098 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.867499 28894 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.881841 29068 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.883172 28894 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:34.882395 28859 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.886449 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:34.887513 23321 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:34.878939 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.905539 23390 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:15:34.905628 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:34.908071 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:34.868412 29105 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:34.924901 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86431.5us
I20260709 12:15:34.921096 23395 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:15:34.921573 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45611.7us
I20260709 12:15:34.863566 29096 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:15:34.923902 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:34.931864 24013 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:34.936724 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.942610 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:34.944710 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:34.923979 28970 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:34.947376 24009 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:34.930099 29105 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:34.986368 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:34.906440 28770 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:15:34.986968 29056 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:34.884240 29009 consensus_queue.cc:1175] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2778b449761a46708f2f643b1b5b5854" responder_term: 14 status { last_received { term: 14 index: 13 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:34.943769 29037 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.001024 28673 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:35.000844 29037 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.003182 29037 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:34.990520 28997 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.922905 29112 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.005529 28997 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:35.006291 28042 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.009121 28042 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:34.993523 29098 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:34.996928 23468 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:35.009732 29098 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:34.994225 29068 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:34.994664 29134 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:15:34.995101 28970 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:34.996204 29128 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
W20260709 12:15:34.996912 29127 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:15:35.024215 29105 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:35.025031 29129 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:15:34.966228 28842 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:15:35.025903 28842 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:34.999322 28875 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:35.003098 28792 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.026942 28842 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.992755 28325 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:34.974851 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 67964.3us
I20260709 12:15:34.955451 29115 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:35.030308 29115 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.930316 29110 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:34.948019 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.018940 23461 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
W20260709 12:15:35.013401 23436 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.527s (SENT) [suppressed 23 similar messages]
I20260709 12:15:34.923475 29118 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.020336 28923 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.023361 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:35.040433 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 116594us
I20260709 12:15:35.141233 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:35.142447 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:35.143500 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:35.125833 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.145259 29010 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.145843 29010 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:35.149555 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.146481 28997 raft_consensus.cc:2764] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:15:35.150044 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:35.150779 29010 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.121359 28970 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:35.151163 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:35.133358 29100 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:35.151791 23390 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:35.148448 28923 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.152494 29105 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.131271 29056 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:35.114720 29135 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.260s	user 0.000s	sys 0.002s
I20260709 12:15:35.154026 23466 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:15:35.155354 23465 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:35.156876 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.149709 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.050140 28325 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:35.113149 29140 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:15:35.115267 29097 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:34.966885 29119 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:15:35.050590 29110 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:35.104969 29043 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:35.139467 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.087385 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.149355 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:35.151374 28792 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:34.947538 28859 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:15:35.106132 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 158215us
I20260709 12:15:35.182264 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.157018 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.158074 23461 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:35.010327 29098 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:35.185719 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:35.164093 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:35.164481 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.186722 24007 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:35.177448 23388 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:35.184124 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.189497 23321 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:35.177299 23468 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:35.191763 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.153241 29056 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.196224 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:35.193666 24010 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:35.198127 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.198675 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.199473 23318 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:35.172923 29110 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:35.206470 28997 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:35.209116 28997 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:35.209172 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.209983 23313 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:35.210093 28997 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.210819 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.211894 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.212911 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.213508 29147 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.213614 23313 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:35.194734 28859 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:35.214681 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.214383 29147 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.210112 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:35.216311 23465 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:35.214380 29126 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:35.216939 29147 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:35.217753 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.218531 23314 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:35.219254 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.215848 28859 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.220220 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.221573 28770 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:35.224308 28940 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:35.224819 28940 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:35.225926 29111 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:35.226533 29111 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.228726 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:35.207816 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.231076 23465 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:35.231181 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.232431 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.221259 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.237136 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.238868 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.239849 23465 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:35.213675 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.151922 29100 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.251988 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:35.217777 23313 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:15:35.253403 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 240043us
I20260709 12:15:35.248975 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.251523 24009 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:35.259958 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.254428 23312 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:35.270432 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.271517 23318 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:35.273175 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.274593 23321 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:35.278817 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.280571 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.191905 28970 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:15:35.282550 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.287498 23321 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:35.284531 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:35.283612 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.295025 23388 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:35.298887 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:35.299904 23314 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:35.293004 23313 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:35.297700 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.303546 24016 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:35.301064 23313 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:35.308219 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.331466 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:35.308503 23313 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.342067 23314 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:35.341856 23313 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:35.345619 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89388.7us
I20260709 12:15:35.350076 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.352756 24009 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:35.359647 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.297982 28940 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.372331 23393 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:35.375612 23392 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:15:35.348011 29037 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:15:35.349639 29144 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:35.383438 23390 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:35.384446 24009 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:15:35.404328 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.428498 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.383670 29097 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.481019 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.468361 24008 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:15:35.482950 29154 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:15:35.483004 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.483228 28842 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:35.439313 29149 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.245s	user 0.005s	sys 0.005s
I20260709 12:15:35.529822 28842 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.531056 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.531816 29100 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:35.532020 29130 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.740s	user 0.004s	sys 0.005s
I20260709 12:15:35.532096 28842 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.532534 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.532792 28842 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:35.485101 29148 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:15:35.487703 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.534400 28673 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.535107 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.384017 28997 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.428447 28940 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:35.535797 28997 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.526443 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.537783 23390 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:35.533217 23396 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:35.538149 28997 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.534894 28673 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.525336 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.540477 28673 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.534217 23313 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:15:35.541935 23313 pending_rounds.cc:85] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 6
I20260709 12:15:35.542162 23313 pending_rounds.cc:107] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:15:35.542430 23313 raft_consensus.cc:2889] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:35.544031 29165 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:35.544605 29165 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:35.546674 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 211611us
I20260709 12:15:35.547451 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:35.549382 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.550076 23313 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:35.556993 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.558082 23313 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:35.520085 29056 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:35.587841 29144 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:35.596798 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 301710us
I20260709 12:15:35.555154 28964 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:35.623270 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:35.623829 29132 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.777s	user 0.002s	sys 0.000s
I20260709 12:15:35.624146 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.551038 29100 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:15:35.625557 28964 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.626005 28964 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:35.626590 28932 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:35.626835 28673 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:35.627127 28932 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:35.627295 28673 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.627727 28673 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:35.628165 28932 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.630626 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:35.595377 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:35.608745 29156 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 12:15:35.635020 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46734.3us
I20260709 12:15:35.649932 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.651542 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.652034 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.652150 23387 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:35.652639 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:35.620339 29111 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:35.658289 23394 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:35.659124 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.659799 23466 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:35.607183 29025 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:35.556758 28970 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:15:35.533347 28842 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.653132 23389 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:35.640096 28875 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:15:35.661412 23392 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:35.662055 23396 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:35.619289 28923 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:35.662467 23465 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:35.662822 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.671316 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:35.671697 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.663825 23395 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:35.664412 23393 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:15:35.661684 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:35.674860 23470 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:35.667922 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:35.677862 23313 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:35.679226 24008 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:35.641283 29038 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.685948 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.676657 28959 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:35.735925 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68084.3us
I20260709 12:15:35.743429 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:35.744751 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.596010 29158 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:15:35.739482 23321 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:35.622664 29144 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.665139 28842 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.746793 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.670293 29068 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:15:35.756419 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:35.759315 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 209099us
I20260709 12:15:35.742236 29043 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:35.761237 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.743047 28959 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:35.746325 29146 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:35.763837 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.765415 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.766350 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.763324 29037 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:35.769783 29161 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 12:15:35.761132 29169 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:35.772603 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:35.782629 23321 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:35.703460 29038 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.762305 28970 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:35.771726 29144 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:35.683878 29165 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.792593 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:35.774178 29155 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:15:35.848860 28673 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.766680 28959 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:35.851380 28673 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.842574 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:35.852577 24016 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:35.791224 28970 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:35.855753 28970 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:35.774133 29110 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.003s
I20260709 12:15:35.766659 29056 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:35.776225 29159 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:35.845755 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:35.845876 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:35.858814 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 92504.7us
W20260709 12:15:35.881386 29170 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:15:35.865485 29173 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:35.868487 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:35.873725 29118 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:35.869904 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:35.888854 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.883632 29173 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.889889 29173 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:35.892431 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:35.896772 24011 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:35.898243 29173 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.899567 29038 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:35.901099 29159 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.901696 29159 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:35.902550 29105 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:35.903084 29105 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:35.871083 29157 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.386s	user 0.001s	sys 0.001s
I20260709 12:15:35.774133 29068 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:15:35.901196 29173 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:35.907502 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 933) took 1118 ms. Trace:
W20260709 12:15:35.908025 29133 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:35.904506 29105 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.909245 29105 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:35.909925 29098 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.692s	user 0.000s	sys 0.011s
I20260709 12:15:35.910385 29098 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.912564 29098 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:35.916062 29152 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.447s	user 0.000s	sys 0.004s
W20260709 12:15:35.870452 29160 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.384s	user 0.000s	sys 0.002s
I20260709 12:15:35.908052 24014 rpcz_store.cc:276] 0709 12:15:34.788683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.827678 (+ 38995us) service_pool.cc:224] Handling call
0709 12:15:35.892113 (+1064435us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b5800085298. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:15:35.907411 (+ 15298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5869824}
W20260709 12:15:35.928561 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 82799.1us
I20260709 12:15:35.929993 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.942346 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:35.943285 23470 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:35.924611 29038 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:35.926883 29043 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:35.949709 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:35.950754 23464 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:35.947039 29056 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:35.954803 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.893232 28964 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.879876 29110 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.955448 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.960494 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.955137 28673 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.955503 29068 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:35.953301 29169 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:35.966650 23362 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.968045 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.973238 28042 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:35.965785 29118 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:35.974090 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:35.962538 29173 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:35.954945 29174 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.962663 29098 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:35.973664 29167 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:35.974488 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.969760 29038 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:35.975378 29169 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.986773 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.975786 29118 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:35.980346 28929 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:35.969537 29146 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 12:15:35.988674 29115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.014s	sys 0.009s
I20260709 12:15:36.010428 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.973069 28673 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:35.954689 29056 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:35.989337 29173 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:36.041826 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 249342us
W20260709 12:15:36.035485 29115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.014s	sys 0.011s
I20260709 12:15:36.043303 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:36.031540 28929 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.028142 29098 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:36.021843 29118 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:36.045377 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:35.983037 29174 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.037184 29126 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:35.990728 28940 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:36.057924 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:36.059958 24010 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:36.060056 23396 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:36.066821 28673 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.067790 29126 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.069976 29126 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.071563 28964 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.071766 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.072162 29068 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:36.072590 29068 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:36.072969 29068 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:36.074178 29110 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:36.075779 23468 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:15:36.076651 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.077190 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:36.077737 29097 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:35.990631 29043 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.078385 29115 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:36.079763 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.079383 28959 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.082094 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36042 (request call id 2800) took 1355 ms. Trace:
I20260709 12:15:36.084110 23320 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:36.084406 23321 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:36.084584 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:36.100780 23463 rpcz_store.cc:276] 0709 12:15:34.726796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.726945 (+   149us) service_pool.cc:224] Handling call
0709 12:15:36.081906 (+1354961us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001883d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:36.082006 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291927296}
I20260709 12:15:36.085655 23316 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:15:36.086650 29162 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:15:36.059784 23395 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:36.093803 23470 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
W20260709 12:15:36.094403 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84106.7us
I20260709 12:15:36.112681 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:35.989558 29038 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.116205 29038 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:36.050797 28929 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:36.107306 29111 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.117877 29111 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.120326 29111 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:36.123751 28959 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.051906 29177 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:36.128993 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:36.130903 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53785.8us
I20260709 12:15:36.131454 23464 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:36.129868 24012 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:36.136897 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:36.103010 28997 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:36.152709 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:36.154590 23395 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:36.155647 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.156304 24010 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:36.158203 23464 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:36.160354 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.160988 23463 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:36.161374 23470 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:36.164019 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.164856 23468 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:36.171064 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:36.165475 29173 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:36.171934 23315 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:36.132730 28940 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.175395 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.130816 29177 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:36.151026 29110 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:15:36.199093 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 92281.8us
I20260709 12:15:36.192464 23387 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:36.150069 29167 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:36.194444 24010 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:36.177106 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:36.197268 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.202082 23316 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:36.198081 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.196242 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.203286 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.203287 24012 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:36.202167 23465 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:15:36.204236 23392 raft_consensus.cc:2410] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 18 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:15:36.204722 23396 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:36.207942 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.208518 23461 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:36.214063 23321 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:36.214159 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:36.207311 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.176726 29173 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:36.237476 23465 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:36.238229 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65355.9us
W20260709 12:15:36.239282 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84107us
I20260709 12:15:36.238420 28042 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:36.191676 23468 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:36.245674 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:36.245682 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.199934 29177 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:36.247076 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.252628 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.252177 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.252416 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.222559 28972 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:36.198671 29037 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:15:36.268867 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.269819 23387 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:36.242329 29173 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:36.196208 29181 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 12:15:36.276176 23396 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:36.277271 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:36.270283 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 67831.1us
I20260709 12:15:36.280308 23461 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:36.270498 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.199347 28940 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.282819 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.200994 28959 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:15:36.205709 28964 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:15:36.285836 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2029) took 1076 ms. Trace:
I20260709 12:15:36.252602 29010 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.199586 29180 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:15:36.242592 29176 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.246687 29174 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.259088 29168 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.295763 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.299880 23387 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:36.210044 28953 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:15:36.236507 29105 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:15:36.246635 29098 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:15:36.301447 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.286423 23312 rpcz_store.cc:276] 0709 12:15:35.209690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:35.294890 (+ 85200us) service_pool.cc:224] Handling call
0709 12:15:35.302007 (+  7117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:35.302331 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:35.302364 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:35.310020 (+  7656us) log.cc:844] Serialized 26 byte log entry
0709 12:15:35.391176 (+ 81156us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:35.490342 (+ 99166us) log.cc:844] Serialized 14 byte log entry
0709 12:15:35.490645 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:35.490874 (+   229us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:36.086033 (+595159us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:36.272694 (+186661us) raft_consensus.cc:1685] finished
0709 12:15:36.285579 (+ 12885us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:36.285758 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8338,"prepare.queue_time_us":111810,"raft.queue_time_us":4366,"raft.run_cpu_time_us":1524,"raft.run_wall_time_us":8364,"spinlock_wait_cycles":57252864,"thread_start_us":80743,"threads_started":1,"wal-append.queue_time_us":427958}
I20260709 12:15:36.249117 28854 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:15:36.245357 29108 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:15:36.252894 28842 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:15:36.326978 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.314725 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:36.260429 29159 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.338627 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.356022 23396 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:36.330334 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 86610.8us
I20260709 12:15:36.339324 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:36.358723 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:36.340619 29188 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:15:36.341326 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 101458us
I20260709 12:15:36.357620 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:36.364576 23465 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:36.364863 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.365782 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.364565 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.378222 23461 raft_consensus.cc:2410] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:15:36.378710 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.330217 29181 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.379506 29181 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:36.380914 29185 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:15:36.381901 29181 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.382555 29190 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:36.383049 29190 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.383432 29190 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.389462 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.390235 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.370127 29202 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.302453 29010 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.390868 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:36.363166 29146 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:36.365931 28923 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:36.330469 29174 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:15:36.311789 29182 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 12:15:36.442109 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 941) took 1077 ms. Trace:
I20260709 12:15:36.465209 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.301371 29176 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.484246 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.365988 28997 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:36.366398 29143 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:15:36.391875 23389 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:36.439034 28923 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.407711 29181 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:36.451625 24011 rpcz_store.cc:276] 0709 12:15:35.364348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:35.373840 (+  9492us) service_pool.cc:224] Handling call
0709 12:15:35.377583 (+  3743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:35.897174 (+519591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:36.420036 (+522862us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54004afbd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:36.442025 (+ 21989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192892672}
W20260709 12:15:36.479399 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 198807us
I20260709 12:15:36.495642 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.410444 23396 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:36.311347 29175 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:15:36.411173 29146 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.308745 29168 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.414224 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.449838 28859 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:36.419016 29037 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:36.402017 29068 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.500799 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:36.390486 28929 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.490105 28997 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.509404 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:36.522495 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 114187us
I20260709 12:15:36.497014 28923 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:36.495949 29189 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.496377 29098 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:36.494956 29181 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:36.541472 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.543938 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:36.516185 29146 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:36.450366 29202 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.538512 29037 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.548655 29192 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:36.551296 29211 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:36.552810 28940 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:36.554230 28929 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.554780 23390 raft_consensus.cc:2368] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:36.545001 28997 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.558000 29186 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:36.558208 29174 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:15:36.559028 29150 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.131s	user 0.014s	sys 0.000s
I20260709 12:15:36.562834 23388 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:36.545240 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.584403 28875 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:36.567281 23469 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:15:36.573575 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 194943us
W20260709 12:15:36.573863 24013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1183) took 1631 ms (client timeout 1968 ms). Trace:
I20260709 12:15:36.580992 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.583290 23396 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:36.597649 29193 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:36.598953 29118 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 12:15:36.569496 28964 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:15:36.565560 23394 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:15:36.625237 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80075.6us
I20260709 12:15:36.625356 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.626398 23461 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:36.603346 23392 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:15:36.622491 24013 rpcz_store.cc:269] 0709 12:15:34.942960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:34.943955 (+   995us) service_pool.cc:224] Handling call
0709 12:15:36.573692 (+1629737us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540011d5d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:36.573810 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415597440}
I20260709 12:15:36.570617 28959 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:15:36.581694 29169 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:15:36.602831 29192 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.580578 29180 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:15:36.633700 29216 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:36.612783 29147 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.637493 29159 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:36.636781 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.641601 29147 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.643051 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.644105 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:36.646103 29175 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:36.646793 29175 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:36.647305 23461 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:36.637236 29176 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.638352 29192 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:36.647420 29175 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:36.636296 29215 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:36.653853 29215 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.655901 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.656159 29215 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.645092 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:36.658532 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:36.658982 29202 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:36.659344 29175 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:36.659978 29037 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.660127 29175 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:36.660291 29168 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.661128 29068 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.661880 29208 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:36.662101 29010 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.622362 29181 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.664427 29181 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:36.668512 28923 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.670794 28972 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:36.671319 28972 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:36.672108 28970 consensus_meta.cc:287] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.816s	user 0.011s	sys 0.000s
I20260709 12:15:36.672533 28970 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.675078 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.677109 29180 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:36.677340 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.677649 29180 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:36.614539 28875 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:36.688622 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:36.568527 29189 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.637929 29193 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:36.696498 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:36.713080 24012 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:36.715130 28875 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.710547 29180 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.670789 28923 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:36.693022 29214 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.058s	user 0.001s	sys 0.003s
I20260709 12:15:36.718236 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:36.720119 24010 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:36.717814 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.711501 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:36.725481 23394 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:36.726598 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.727257 23394 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:36.730340 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.731025 23396 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:36.720484 24010 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:36.635604 29118 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.733600 23392 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:15:36.711889 29098 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:36.735597 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:36.733668 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:36.743110 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 183699us
I20260709 12:15:36.714423 29193 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:36.719781 28859 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:36.691984 29173 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.677457 29190 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.750821 23392 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:36.755074 29118 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:36.757395 29213 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:15:36.746187 29208 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:36.763063 28859 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.736783 28875 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:36.809343 23465 raft_consensus.cc:1240] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:15:36.811744 28859 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.736987 29159 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:36.743897 24010 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.749832 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.831604 23461 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:36.831975 23396 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:36.832736 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.740316 29180 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:36.746639 29098 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:36.813236 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:36.837371 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.837785 24010 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:36.844358 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:36.844821 23462 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:36.811748 29177 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 12:15:36.903832 29177 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.907186 29146 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.964389 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:36.814098 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:36.844925 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:36.858779 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:36.844936 28970 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.861205 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.861526 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:36.837419 29208 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:36.983938 23388 raft_consensus.cc:2410] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 12:15:36.887246 29221 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:15:36.888993 29190 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:36.890107 29098 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:36.894301 29159 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:37.002105 29208 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:36.896423 29173 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:36.902559 29201 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.453s	user 0.013s	sys 0.000s
I20260709 12:15:36.983031 23469 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:37.005120 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.013309 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:37.018126 23394 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:37.018770 28673 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.022585 23396 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:37.025000 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:37.026285 23396 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:37.027096 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:37.024381 28673 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.029729 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.030215 28673 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.030902 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.031205 29068 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:37.031638 29068 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:37.031996 29068 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:37.030993 29037 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:37.032859 29037 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:37.033510 29037 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:37.035665 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.036935 23320 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:37.040024 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:37.040705 23391 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:37.042029 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.042769 23312 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:37.043078 23319 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:37.043242 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.044332 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.045594 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.046439 23317 raft_consensus.cc:2410] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:15:37.050511 23394 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:36.990674 29192 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.024672 29186 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.069005 23391 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:37.022135 29174 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:37.003094 29159 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.023176 29181 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.057271 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.057282 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.070104 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:37.070981 23469 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:37.079208 23391 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:37.064219 29200 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.694s	user 0.005s	sys 0.000s
I20260709 12:15:37.085641 29173 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:37.086032 29174 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.076081 29186 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.092375 23318 raft_consensus.cc:2368] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:37.063016 29189 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.111495 24010 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:37.027746 28997 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.083039 29177 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.146754 23320 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:37.121070 29173 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:37.083029 29181 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.066859 29193 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:37.158367 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.152720 28932 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:37.166533 23395 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: [15.13-17.14]
I20260709 12:15:37.186744 23395 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:37.168920 28932 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:37.193897 29158 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:37.189532 29146 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.169476 28972 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.243772 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.245748 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.246326 23317 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:37.246027 28959 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:37.246219 29188 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:37.256323 28959 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:37.212785 29173 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.256620 29188 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.256532 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.233219 29192 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.269325 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.270499 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:37.256608 29230 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:15:37.278095 29181 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.266752 29174 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.233489 28932 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.281153 28932 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:37.258548 29186 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.257287 28959 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.294380 28959 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:37.335734 29233 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:37.335911 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1010) took 1174 ms. Trace:
I20260709 12:15:37.336392 23389 rpcz_store.cc:276] 0709 12:15:36.161980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.162519 (+   539us) service_pool.cc:224] Handling call
0709 12:15:36.388220 (+225701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:36.489733 (+101513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.335653 (+845920us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54001978d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:37.335837 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518300032}
I20260709 12:15:37.397605 28770 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.398327 28770 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:37.404654 29231 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:15:37.406121 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2634) took 1033 ms. Trace:
I20260709 12:15:37.410003 23468 rpcz_store.cc:276] 0709 12:15:36.372974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.408752 (+ 35778us) service_pool.cc:224] Handling call
0709 12:15:36.409100 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:37.405186 (+996086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:37.405222 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:37.405270 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:37.405306 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.405876 (+   570us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:37.405922 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.406062 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290266496}
W20260709 12:15:37.418838 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 372792us
I20260709 12:15:37.421399 23465 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:37.423134 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:37.425150 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.427205 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:37.427904 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.430401 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.436170 29188 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.436928 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.438062 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.438800 23465 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:37.443327 23434 leader_election.cc:400] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:37.443954 23317 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:37.444059 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:37.444566 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:37.444918 23392 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:37.445178 23469 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:37.445658 23434 leader_election.cc:403] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:15:37.446569 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:37.447198 23394 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
W20260709 12:15:37.450081 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.453114 24014 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:37.453449 24014 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:37.454295 24014 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.455266 24014 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:37.457394 29068 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.458580 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:37.459178 24015 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:37.437125 29165 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:15:37.461556 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:37.462625 29146 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.463119 29146 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.465158 29146 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.433713 28770 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.460167 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.458002 24010 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:37.506886 24008 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:37.496678 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:37.471063 29108 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:15:37.497607 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.497951 23321 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:37.507853 23394 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:37.509861 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.510478 24013 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:37.501184 23392 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:37.433353 28972 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.504158 23982 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:37.505236 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.522461 24009 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:15:37.525990 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:37.508205 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65023.1us
I20260709 12:15:37.519464 23360 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:37.460121 29068 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.536737 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.497152 29111 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.534283 23318 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:37.534827 23315 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:37.513607 29202 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.558562 23469 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:37.568558 23470 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:37.564002 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.545902 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:37.571980 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:37.590449 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:37.575439 24013 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:37.592370 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:37.593502 24013 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:15:37.577785 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.577266 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.594525 23468 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
W20260709 12:15:37.578897 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.579334 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.567405 29165 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:37.581141 23362 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:37.550150 29203 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:37.602880 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.588333 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.584626 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:37.604809 23466 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:37.587203 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:37.605732 23317 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:37.586720 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:37.550685 28972 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:37.599257 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1303) took 1038 ms. Trace:
I20260709 12:15:37.603480 28042 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:37.611927 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:37.612040 28042 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.601732 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:37.615310 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 2176) took 1405 ms. Trace:
I20260709 12:15:37.608716 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.615833 23390 rpcz_store.cc:276] 0709 12:15:36.209661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.211558 (+  1897us) service_pool.cc:224] Handling call
0709 12:15:37.615122 (+1403564us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b54002c8154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:37.615256 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703029120}
I20260709 12:15:37.596726 29110 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.625313 24014 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:15:37.575738 29202 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.651729 28042 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.608119 29175 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:37.573063 29111 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.629635 24014 pending_rounds.cc:85] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 11
I20260709 12:15:37.627303 23394 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:37.609908 24011 rpcz_store.cc:276] 0709 12:15:36.560604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.560753 (+   149us) service_pool.cc:224] Handling call
0709 12:15:36.573295 (+ 12542us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:36.583156 (+  9861us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:36.599461 (+ 16305us) log.cc:844] Serialized 14 byte log entry
0709 12:15:37.449625 (+850164us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b18002f8d80. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0ad54858f 00007ff0a6587571 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd
0709 12:15:37.466543 (+ 16918us) log.cc:844] Serialized 14 byte log entry
0709 12:15:37.466765 (+   222us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:37.467342 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.467516 (+   174us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:37.467562 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.467661 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:37.599029 (+131368us) raft_consensus.cc:1685] finished
0709 12:15:37.599059 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.599197 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":790964,"prepare.queue_time_us":19344,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":2709,"raft.queue_time_us":780481,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":284,"spinlock_wait_cycles":5969664,"thread_start_us":89168,"threads_started":1,"wal-append.queue_time_us":118654}
I20260709 12:15:37.600455 29159 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:37.603173 29196 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:37.608280 29146 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:37.668919 29146 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:37.669374 29146 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:37.670270 29202 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:37.671909 23436 leader_election.cc:336] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.586s (SENT)
I20260709 12:15:37.621668 29147 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.617403 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:37.676755 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:37.666847 24014 pending_rounds.cc:107] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:15:37.677676 24014 raft_consensus.cc:2889] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:37.679039 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:37.615936 29203 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:37.682360 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65036.2us
I20260709 12:15:37.684724 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:37.654032 23469 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:15:37.630497 29209 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:37.607575 29105 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:15:37.630234 29108 raft_consensus.cc:2764] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:15:37.685624 23469 pending_rounds.cc:85] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 8
I20260709 12:15:37.688046 23469 pending_rounds.cc:107] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:15:37.688325 23469 raft_consensus.cc:2889] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:37.667397 29196 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:37.690708 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:37.691617 29203 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.692286 29111 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.674211 29159 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.652832 29126 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 12:15:37.703889 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 81449.1us
I20260709 12:15:37.704725 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:37.653169 29110 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.706334 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:37.686491 29202 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:15:37.695549 29198 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 1.324s	user 0.002s	sys 0.011s
I20260709 12:15:37.694877 29196 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:37.715127 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:37.675832 29175 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.712291 29202 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:37.711705 29209 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:37.716881 29144 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:37.727924 29175 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.732354 23436 leader_election.cc:400] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:37.732842 23436 leader_election.cc:403] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:37.734913 28997 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.742801 28964 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:37.743378 28964 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.745693 28964 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.746918 24008 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:37.719473 29206 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:37.749410 28042 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:15:37.750385 28940 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:37.773168 29159 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.774192 24014 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:15:37.724814 29202 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:37.775395 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 60210.6us
I20260709 12:15:37.747668 29209 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:37.757979 29173 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:37.776656 29247 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:15:37.775180 29175 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:37.777788 28770 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:37.777961 29144 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:37.823894 29250 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:15:37.843627 29025 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:37.818509 29259 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:37.801232 28964 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:37.892537 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 198524us
I20260709 12:15:37.886621 29187 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:37.809623 29173 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.890442 29025 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.886443 28875 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:37.902870 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.914359 23387 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:37.904587 29209 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:37.922948 29206 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:37.923537 29206 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:37.885124 28770 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:37.914062 28959 consensus_meta.cc:287] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.009s
I20260709 12:15:37.931557 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:37.930132 23436 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.875s (SENT)
I20260709 12:15:37.886029 29175 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:37.900936 29220 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:15:37.932354 23396 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:37.934578 28770 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.938241 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:37.938891 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:37.947016 23393 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:15:37.897212 28964 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:37.947800 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 152922us
I20260709 12:15:37.953187 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:37.953081 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.932288 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:37.926582 28875 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:37.960376 29256 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:15:37.960981 28964 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.962605 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.965804 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:37.941139 29110 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:37.961790 29105 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:37.969403 23436 leader_election.cc:336] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Timed out: RequestConsensusVote RPC to 127.21.150.196:40405 timed out after 1.968s (SENT)
I20260709 12:15:37.971529 29173 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:37.974367 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1210) took 1257 ms. Trace:
I20260709 12:15:37.961094 28875 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:37.965391 29098 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:38.008049 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:38.034128 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:38.034795 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 75891.5us
I20260709 12:15:37.996310 29110 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:38.009594 24012 rpcz_store.cc:276] 0709 12:15:36.716721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.716850 (+   129us) service_pool.cc:224] Handling call
0709 12:15:36.717183 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:37.467961 (+750778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:37.468005 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:37.468534 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.595226 (+126692us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:37.665166 (+ 69940us) log.cc:844] Serialized 14 byte log entry
0709 12:15:37.665582 (+   416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:37.665655 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:37.974162 (+308507us) raft_consensus.cc:1685] finished
0709 12:15:37.974184 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:37.974309 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":750527,"prepare.queue_time_us":73488,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":759571,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6729,"thread_start_us":126330,"threads_started":1,"wal-append.queue_time_us":212617}
I20260709 12:15:37.967973 29144 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.040647 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:38.041324 29254 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:15:38.042040 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.043579 23388 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:38.044371 29206 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:38.045914 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 80176.2us
I20260709 12:15:37.990588 29264 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:15:38.036322 29173 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:38.036453 29266 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:38.081096 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 167915us
I20260709 12:15:38.005505 29146 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.084436 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.089499 28042 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:38.074581 29098 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:38.097915 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:38.091214 29245 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.439s	user 0.004s	sys 0.001s
I20260709 12:15:38.045845 29110 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.097391 29203 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:38.097685 29220 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:38.096230 28770 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:38.103519 29264 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:38.159904 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 76664.5us
I20260709 12:15:38.163688 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:38.151625 29190 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.120301 29267 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:38.143312 29274 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:38.146541 29175 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.091621 29266 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.183943 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1208) took 1486 ms. Trace:
I20260709 12:15:38.148519 29173 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:38.188936 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 99682.3us
I20260709 12:15:38.103237 29098 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:38.194351 29262 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:15:38.194015 29173 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.194494 29286 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.195159 29203 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:15:38.209224 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 88185.5us
I20260709 12:15:38.210667 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.193209 24016 rpcz_store.cc:276] 0709 12:15:36.697877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:36.697982 (+   105us) service_pool.cc:224] Handling call
0709 12:15:36.717766 (+ 19784us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:37.466663 (+748897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:37.466694 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:37.467197 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.595561 (+128364us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:37.949966 (+354405us) log.cc:844] Serialized 14 byte log entry
0709 12:15:38.176451 (+226485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.178563 (+  2112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:38.178904 (+   341us) raft_consensus.cc:1685] finished
0709 12:15:38.182691 (+  3787us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:38.183555 (+   864us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":818222,"prepare.queue_time_us":73440,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":803863,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":22266,"thread_start_us":628582,"threads_started":3,"wal-append.queue_time_us":251811}
I20260709 12:15:38.213511 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:38.196646 29265 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:15:38.153301 29220 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.229398 29098 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:38.231132 29266 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.231220 29274 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:38.232331 29278 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:38.216578 29206 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.233862 29264 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:38.234385 29264 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.236603 29264 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:38.237102 29261 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
W20260709 12:15:38.240475 29260 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:15:38.242544 29173 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.244597 29220 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.244792 29173 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:15:38.215601 29286 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.189379 29190 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.248009 29190 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:38.248548 29202 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.249126 29146 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.249699 24011 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
W20260709 12:15:38.250200 29277 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:15:38.256314 29196 consensus_meta.cc:287] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.002s
I20260709 12:15:38.256745 29196 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.216681 29203 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:38.258265 29203 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.259868 29286 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.260983 29159 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:38.240389 29285 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:38.265815 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.266283 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.267813 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.281643 23394 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:38.278491 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:38.263149 28932 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.938s	user 0.019s	sys 0.013s
I20260709 12:15:38.288892 24013 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:38.280742 28875 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.280628 29202 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:38.284889 29285 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:38.307909 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 128309us
I20260709 12:15:38.310541 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.311503 23393 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:15:38.313112 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:38.308898 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:38.315397 28854 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:38.318701 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:38.319201 29268 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 12:15:38.319417 24009 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.319223 28854 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:38.320562 28854 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:38.321897 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:38.322074 29193 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.296774 28875 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.327699 28875 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.328300 29110 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.329608 29119 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:38.330524 29119 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:38.331568 28842 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:38.331773 29119 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:38.331934 28842 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:38.332181 29119 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:38.310587 29175 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.333127 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.340734 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.341329 23462 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:38.333413 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.345834 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.346509 29285 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.347626 23387 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:38.333850 29284 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:38.349792 29284 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:38.350137 29284 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:38.350610 29212 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:38.350463 28842 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.351702 28842 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:38.352977 28875 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.353468 28875 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.354597 28042 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:15:38.354995 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 141596us
I20260709 12:15:38.354959 23462 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:38.348938 28932 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:38.348941 29175 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.351717 29212 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:38.358191 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:38.361276 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.363605 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.388456 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.376884 24009 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:38.378935 28042 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:38.391817 28042 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.390657 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:38.402384 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.403671 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.403823 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.404305 23465 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:38.404665 24010 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:15:38.405200 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.406365 23464 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:15:38.407490 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:38.408257 23461 raft_consensus.cc:2393] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:38.409027 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.409780 24007 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:38.411659 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.411760 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.414022 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.412833 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.414785 24012 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:38.416940 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.417709 23465 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:38.417812 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:38.412117 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.420034 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.421131 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.393700 29111 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:38.385622 28770 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:38.421934 29289 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:15:38.422526 23467 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:38.423202 28997 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:38.421926 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:38.423789 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.418771 24007 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:38.423869 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:38.371147 29025 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:38.368831 29212 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:38.425397 24008 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:38.425400 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:38.426407 29212 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.426908 29105 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:38.427011 23319 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.427412 29105 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:38.427699 29212 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:38.423941 23466 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:38.397033 29206 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:15:38.410161 28854 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.367738 29187 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:38.432478 28854 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.389830 29202 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:38.434988 29169 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:38.369065 29175 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.440579 29165 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:38.441810 23319 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:38.405536 29196 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:38.366888 28875 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.445166 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.445835 28970 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:38.446676 23393 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: [9.10-11.11]
I20260709 12:15:38.393177 29119 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.448148 29119 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:38.418704 28964 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:38.448961 28964 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.449736 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:38.450606 29270 log.cc:927] Time spent T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.351s	user 0.002s	sys 0.000s
I20260709 12:15:38.413057 29147 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:15:38.451182 28964 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.453061 28042 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.457892 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.458524 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.458608 23319 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:38.459121 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.459458 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:38.460192 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:38.409188 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:38.484774 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:38.456544 28997 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:38.469686 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.407775 29108 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:38.497925 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.485597 28970 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:38.489157 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.421950 29111 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.461652 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.548318 23313 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:38.526291 23320 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:38.462512 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.551841 23317 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
W20260709 12:15:38.505338 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 96216.7us
I20260709 12:15:38.483284 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:38.464996 28964 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:38.535175 23314 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
W20260709 12:15:38.510576 29271 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.358s	user 0.002s	sys 0.000s
I20260709 12:15:38.567049 29108 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.569597 29108 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.570905 23314 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:38.574662 23461 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:38.577652 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:38.513461 23316 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:38.587266 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1548) took 1070 ms. Trace:
W20260709 12:15:38.581251 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.590759 29126 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.588032 23315 rpcz_store.cc:276] 0709 12:15:37.517086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:37.611664 (+ 94578us) service_pool.cc:224] Handling call
0709 12:15:37.625794 (+ 14130us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:37.654675 (+ 28881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:37.654708 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:37.671162 (+ 16454us) log.cc:844] Serialized 26 byte log entry
0709 12:15:37.845086 (+173924us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b1800318fa0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:38.035653 (+190567us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:38.035702 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.035785 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:38.509916 (+474131us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:38.586986 (+ 77070us) raft_consensus.cc:1685] finished
0709 12:15:38.587010 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:38.587169 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25247,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":14360,"spinlock_wait_cycles":54809728,"thread_start_us":190354,"threads_started":1,"wal-append.queue_time_us":419657}
I20260709 12:15:38.591406 29126 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.592245 23314 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:38.593068 29105 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.593636 29126 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.559935 29111 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.594108 23362 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.595142 29126 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:38.595283 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:38.596253 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.597060 29286 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:38.597581 29286 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:38.597704 29203 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:38.598469 29286 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.564517 29193 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:15:38.565243 28970 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:38.485787 28875 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:15:38.602409 28970 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.603482 28875 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:38.603890 28875 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:38.606308 29267 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.606848 29267 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.607705 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:38.608402 24013 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:38.608861 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:38.609289 28842 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:38.612375 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.608304 28964 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:38.616533 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.604732 28875 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.524726 28940 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:38.630682 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:38.632007 23396 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:15:38.501073 29169 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 12:15:38.635454 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:38.637352 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:38.637809 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 143449us
I20260709 12:15:38.594120 29105 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:38.655861 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:38.659256 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:38.660720 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:38.661954 24015 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:38.664201 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.665589 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:38.668277 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:38.670001 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:38.566323 29284 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.630623 29108 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:38.622982 28964 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:38.631240 28940 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:38.676901 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.670758 28875 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:38.673722 23316 raft_consensus.cc:2955] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 5 to 11, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:38.684713 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.677865 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:38.689621 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:38.689834 23396 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.690183 23396 raft_consensus.cc:614] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:38.672026 29279 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:38.673763 29108 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:38.694077 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 135056us
I20260709 12:15:38.687958 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.666215 29146 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.673451 29097 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:38.666026 29147 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.017s
W20260709 12:15:38.712760 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74022.1us
I20260709 12:15:38.679006 28940 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:38.721403 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.721505 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.722851 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:38.726909 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.719065 29146 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.681412 28997 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.731858 29146 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:38.732618 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.700091 29206 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:38.711594 29108 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:38.741063 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:38.726154 29269 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:38.707227 29279 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.743785 29279 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:38.747371 29175 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.748126 29175 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:38.750852 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:38.753386 29283 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:38.753978 29283 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:38.754508 29283 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:38.757773 29146 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:38.758349 29146 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:38.758870 29146 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:38.764281 29175 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:38.715410 29289 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:38.720295 29144 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:15:38.771360 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52089.5us
I20260709 12:15:38.734009 29110 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:38.799185 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48419.1us
I20260709 12:15:38.738559 29274 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:38.743100 29269 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:38.724380 29220 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:15:38.740477 29097 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:38.734695 29264 consensus_queue.cc:1059] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:15:38.818933 23219 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 221) took 25863 ms (client timeout 15000 ms). Trace:
I20260709 12:15:38.829548 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:38.810997 24014 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:38.822382 29269 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:38.832787 29247 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.090s	user 0.003s	sys 0.000s
I20260709 12:15:38.725788 29296 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 12:15:38.839479 29144 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:15:38.825585 29169 consensus_queue.cc:1243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:15:38.737574 28964 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:38.832408 23219 rpcz_store.cc:269] 0709 12:15:12.938990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:12.940181 (+  1191us) service_pool.cc:224] Handling call
0709 12:15:38.801984 (+25861803us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:34.390908 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:34.391568 (+   660us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:34.391617 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:15:34.391636 (+    19us) tablet.cc:662] Decoding operations
0709 12:15:34.392574 (+   938us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:34.392670 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:34.392710 (+    40us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:15:34.421245 (+ 28535us) tablet.cc:701] Row locks acquired
0709 12:15:34.465791 (+ 44546us) write_op.cc:260] PREPARE: finished
0709 12:15:34.486385 (+ 20594us) write_op.cc:270] Start()
0709 12:15:34.500260 (+ 13875us) write_op.cc:276] Timestamp: P: 1783599334481465 usec, L: 0
0709 12:15:34.510643 (+ 10383us) op_driver.cc:348] REPLICATION: starting
0709 12:15:34.529251 (+ 18608us) log.cc:844] Serialized 3242 byte log entry
0709 12:15:35.145740 (+616489us) op_driver.cc:464] REPLICATION: finished
0709 12:15:35.362114 (+216374us) write_op.cc:301] APPLY: starting
0709 12:15:35.365107 (+  2993us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:35.848765 (+483658us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:35.848927 (+   162us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:35.870706 (+ 21779us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:35.871253 (+   547us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:15:35.871315 (+    62us) write_op.cc:312] APPLY: finished
0709 12:15:35.871669 (+   354us) log.cc:844] Serialized 81 byte log entry
0709 12:15:35.871853 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.871990 (+   137us) write_op.cc:454] Released schema lock
0709 12:15:35.872043 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:35.872488 (+   445us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:35.872516 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":397696,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":197804,"apply.run_cpu_time_us":16774,"apply.run_wall_time_us":518188,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":344178,"prepare.run_cpu_time_us":9092,"prepare.run_wall_time_us":394623,"raft.queue_time_us":118872,"raft.run_cpu_time_us":46066,"raft.run_wall_time_us":132196,"replication_time_us":634928,"thread_start_us":516069,"threads_started":4,"wal-append.queue_time_us":313623,"wal-append.run_cpu_time_us":28032,"wal-append.run_wall_time_us":2030942}]]}
W20260709 12:15:38.864444 29300 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.240s	user 0.000s	sys 0.002s
I20260709 12:15:38.819262 29110 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:38.907420 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 105694us
I20260709 12:15:38.882495 29315 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:38.921149 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:38.910545 28964 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:38.958341 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.968346 23465 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:38.969698 23468 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:38.976769 28940 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:38.978189 24011 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:38.980729 28940 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:38.937392 29266 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:39.037864 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 315060us
I20260709 12:15:39.075350 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:39.048382 29286 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:39.059227 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:38.958670 29173 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:38.989149 29310 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 12:15:38.983860 29147 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:38.937371 28932 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:38.987277 29261 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:15:39.101167 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:38.985567 29304 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 12:15:39.004736 29187 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.002s
W20260709 12:15:39.062634 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.004664 29203 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:39.143954 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:39.105604 29025 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:39.138734 29314 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:15:39.106300 29274 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:39.145318 29302 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:15:38.936197 29315 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:39.145893 29212 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.718s	user 0.008s	sys 0.005s
W20260709 12:15:39.183660 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 225593us
I20260709 12:15:39.115504 29322 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:39.185526 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:39.180757 29308 log.cc:927] Time spent T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.315s	user 0.000s	sys 0.001s
W20260709 12:15:39.181118 29111 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 12:15:39.138904 29147 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.115674 29173 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.108137 29266 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.197757 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:39.110392 29286 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.180387 29165 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:39.215659 29316 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
W20260709 12:15:39.266124 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 525120us
I20260709 12:15:39.267611 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:39.161744 28932 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.146087 29321 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:39.180961 29025 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.277043 29110 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.277810 29110 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.278309 29025 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:39.278257 29110 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.279021 29025 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:39.212206 29111 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.282838 29111 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:39.284695 29283 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.530s	user 0.007s	sys 0.000s
I20260709 12:15:39.286674 29315 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.291035 28932 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.295640 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.309418 29173 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.309914 29283 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.311144 29147 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.312242 29284 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.312686 29110 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.288920 29269 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.313954 29266 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.316772 29269 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:39.299538 29279 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.000s
I20260709 12:15:39.317919 29279 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.319892 29279 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.208104 29305 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:39.302174 29330 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:39.320611 29330 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.323021 29330 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:39.302762 29286 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.324995 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:39.325799 29105 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.002s
W20260709 12:15:39.327121 29108 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.554s	user 0.006s	sys 0.003s
I20260709 12:15:39.327512 29108 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.330107 29328 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:39.330526 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.334508 29283 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.335668 23321 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
W20260709 12:15:39.296634 29126 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.001s
W20260709 12:15:39.306903 29146 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.005s
W20260709 12:15:39.352234 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 365014us
I20260709 12:15:39.346073 29126 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:39.345628 29305 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:39.374744 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 164265us
W20260709 12:15:39.401335 24013 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.004s	sys 0.017s
I20260709 12:15:39.390023 29097 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:15:39.393008 29306 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.582s	user 0.000s	sys 0.003s
I20260709 12:15:39.372578 29108 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.349310 29339 consensus_queue.cc:1059] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:15:39.403146 29097 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:39.351648 29146 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.430258 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:39.384972 29283 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:15:39.450851 23395 raft_consensus.cc:606] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.451318 23395 raft_consensus.cc:614] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:39.447546 29337 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 12:15:39.460824 29336 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 12:15:39.461545 29338 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.462116 29338 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.464380 29338 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.420749 29285 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:15:39.466876 29305 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.468287 29097 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.443457 29108 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:39.469854 29108 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.472019 29108 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:39.473788 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.474768 29325 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:39.475272 29325 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:39.475757 29325 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:39.479023 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2224) took 1113 ms. Trace:
W20260709 12:15:39.481949 23979 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.003s	sys 0.017s
W20260709 12:15:39.484243 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 216705us
I20260709 12:15:39.448551 29169 raft_consensus.cc:3055] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:39.473693 29146 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.479415 23390 rpcz_store.cc:276] 0709 12:15:38.365457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.365605 (+   148us) service_pool.cc:224] Handling call
0709 12:15:38.369855 (+  4250us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:38.385813 (+ 15958us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:38.385845 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:38.407386 (+ 21541us) log.cc:844] Serialized 52 byte log entry
0709 12:15:38.632057 (+224671us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:38.632114 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:38.632207 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:39.392324 (+760117us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:39.478730 (+ 86406us) raft_consensus.cc:1685] finished
0709 12:15:39.478750 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:39.478964 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15544,"raft.queue_time_us":20113,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":19243,"spinlock_wait_cycles":16864640,"thread_start_us":130664,"threads_started":1,"wal-append.queue_time_us":248948}
W20260709 12:15:39.485193 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.008s	sys 0.014s
W20260709 12:15:39.484655 23979 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.004s	sys 0.017s
W20260709 12:15:39.490386 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.008s	sys 0.014s
I20260709 12:15:39.486552 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.485462 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:39.494831 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:39.483269 29097 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:39.488726 29110 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.476428 29317 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:15:39.502707 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.503664 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.489385 29169 raft_consensus.cc:740] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:39.505177 23312 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:39.508160 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.508898 23320 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:39.511695 23436 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:39.511754 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.512609 23317 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:39.518121 23319 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:15:39.518862 23319 pending_rounds.cc:85] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:39.519165 23319 pending_rounds.cc:107] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:15:39.519801 23319 raft_consensus.cc:2889] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:39.499030 29318 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:39.531143 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:39.531947 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.534415 23312 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:39.536391 23313 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:15:39.509698 29173 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:39.539551 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.539383 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.505599 29344 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:39.544467 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.522868 29105 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.544335 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.544930 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.546022 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.549507 23317 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:39.550297 24013 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.546322 23319 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:15:39.550742 23319 pending_rounds.cc:85] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:39.551069 23319 pending_rounds.cc:107] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:15:39.551362 23319 raft_consensus.cc:2889] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:39.551271 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.552917 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.533130 29025 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:39.546520 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.556517 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:39.547142 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.557632 23318 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:39.556521 24008 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:15:39.559243 24015 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.554455 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.560127 23319 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:39.508785 29110 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.563619 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.566545 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.567639 23312 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.543186 29173 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:39.544540 29344 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:39.574783 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.575237 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:39.575855 24015 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.576066 24008 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:39.578760 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:39.582957 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.561883 29146 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:39.606304 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.570209 28997 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.558303 29328 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:39.572875 29322 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:39.608202 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.577491 29348 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:15:39.631929 29322 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:39.632732 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.633849 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.610235 23317 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:39.594265 29349 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:39.636443 23317 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:15:39.636941 23317 pending_rounds.cc:85] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 7
I20260709 12:15:39.637287 23317 pending_rounds.cc:107] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:15:39.637714 23317 raft_consensus.cc:2889] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:39.578289 29344 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:39.639514 23317 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:39.640113 23317 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:39.634811 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.640825 23317 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.642220 23317 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:39.642068 29328 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.644356 29349 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:39.642585 23318 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:15:39.645118 23318 pending_rounds.cc:85] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 12
I20260709 12:15:39.645458 23318 pending_rounds.cc:107] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:15:39.630066 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.645762 23318 raft_consensus.cc:2889] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:39.646343 23319 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:39.647442 23319 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:39.653798 23317 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:39.659017 29110 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:39.549878 29169 consensus_queue.cc:260] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.666420 29169 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:39.670885 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:39.671834 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 134717us
I20260709 12:15:39.578213 29173 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:39.557318 29105 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.670987 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.682746 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.684306 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.677289 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:39.673560 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.687282 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.691406 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.672180 29285 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.696897 24008 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:39.704430 24016 raft_consensus.cc:2410] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:15:39.705933 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:39.703605 24015 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:39.637643 28997 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.715708 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:39.686621 29353 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:39.670033 29146 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.676874 29339 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:15:39.693889 29318 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:39.701242 29025 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:39.697978 29317 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:39.734726 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.724124 29288 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.775894 29322 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.711683 29285 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.720126 29286 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:15:39.794790 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:39.777647 29144 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:39.735534 29025 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:39.730734 29337 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:15:39.730924 29338 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:15:39.788196 29292 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:39.811143 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 97995.2us
W20260709 12:15:39.814266 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 79718.1us
I20260709 12:15:39.791682 29279 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:39.791949 29126 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:39.818627 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:39.820801 23315 raft_consensus.cc:1240] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:39.793109 29105 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:39.827332 23319 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:39.833150 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:39.833508 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.833972 24007 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:15:39.834199 23388 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:39.830137 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.830633 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.828765 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:39.837173 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.837380 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.837740 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:39.837859 23393 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:39.839180 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.826524 29326 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:39.840287 23387 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:39.840368 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.841081 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:39.842376 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.789862 29288 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.844290 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.845808 23392 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:39.846843 23388 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:39.849563 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.846880 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:39.847370 24008 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:39.851307 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:39.856302 24010 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:39.809254 29144 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.846396 29345 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:39.815187 29344 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.856870 24013 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:39.818023 29318 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
W20260709 12:15:39.866956 29354 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:15:39.862324 29025 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.856423 29343 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:39.868522 29285 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:39.869158 29319 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:15:39.870548 29144 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.870801 28972 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:39.828735 29292 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.876142 29345 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:39.876829 29360 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:15:39.838768 29126 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.843873 29352 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.198s
I20260709 12:15:39.878080 28997 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:39.878902 29357 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:15:39.881603 29347 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:15:39.834918 29279 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.890580 23392 raft_consensus.cc:1217] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Deduplicated request from leader. Original: 10.7->[11.8-13.9]   Dedup: 11.8->[13.9-13.9]
I20260709 12:15:39.894120 24012 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:15:39.895228 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80807.9us
W20260709 12:15:39.901527 29356 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:15:39.903195 29328 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:39.912313 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:39.903987 28972 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:39.923074 23392 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:15:39.908726 29345 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:39.932736 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1226) took 1214 ms. Trace:
I20260709 12:15:39.935359 24014 rpcz_store.cc:276] 0709 12:15:38.718282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.718442 (+   160us) service_pool.cc:224] Handling call
0709 12:15:38.718825 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:38.834267 (+115442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:39.932500 (+1098233us) spinlock_profiling.cc:243] Waited 91.8 ms on lock 0x7b540044ff54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:39.932675 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211112704}
I20260709 12:15:39.936815 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.937072 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.932996 24015 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:39.937934 24014 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:39.940825 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.941020 23389 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:39.942801 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:39.945344 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.948743 24015 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.914114 29339 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
W20260709 12:15:39.953238 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 125050us
I20260709 12:15:39.959733 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:39.954071 29313 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:39.961856 29313 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:39.961189 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:39.916589 29325 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:39.941036 29159 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:39.935216 29328 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:39.960604 29352 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:39.965312 29318 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:39.997043 23461 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:39.997187 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.000761 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.002061 29267 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.004815 23466 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:39.970686 28972 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.006669 29126 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:39.972637 29288 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.007352 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.011076 23466 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:40.022293 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.025424 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:40.026285 24007 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:39.973347 29159 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.025998 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.040951 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.053565 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:40.052539 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.054105 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:40.056206 23470 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:40.057986 29279 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:40.062080 29372 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:15:40.065587 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:40.072957 29358 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.245s	user 0.001s	sys 0.001s
I20260709 12:15:40.074748 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1315) took 1497 ms. Trace:
I20260709 12:15:40.122243 24011 rpcz_store.cc:276] 0709 12:15:38.577200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.577348 (+   148us) service_pool.cc:224] Handling call
0709 12:15:38.577661 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:38.978542 (+400881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.074479 (+1095937us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54003741d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:40.074684 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895865728}
W20260709 12:15:40.076151 29361 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:15:40.079933 29173 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.083016 29159 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.083817 23463 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:40.034915 29344 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.087637 29359 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:40.088073 29367 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
W20260709 12:15:40.093684 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:39.994767 29352 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.097365 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1316) took 1518 ms. Trace:
I20260709 12:15:40.094807 29292 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.099561 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.101523 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.096985 29105 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.105046 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.106069 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.158803 23464 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:40.106901 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.096933 29278 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:40.109740 29208 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.109999 29328 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:39.999480 29313 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.114841 29325 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.127389 23461 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:40.137346 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.137521 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.139444 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.186287 23467 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:40.142192 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.149945 23468 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:15:40.153206 24009 rpcz_store.cc:276] 0709 12:15:38.578599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.579026 (+   427us) service_pool.cc:224] Handling call
0709 12:15:38.579445 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:38.579929 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:38.688300 (+108371us) log.cc:844] Serialized 14 byte log entry
0709 12:15:40.012228 (+1323928us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003aff60. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a
0709 12:15:40.018240 (+  6012us) log.cc:844] Serialized 14 byte log entry
0709 12:15:40.018408 (+   168us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:40.024283 (+  5875us) log.cc:844] Serialized 26 byte log entry
0709 12:15:40.024519 (+   236us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:40.024576 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.024668 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:40.097160 (+ 72492us) raft_consensus.cc:1685] finished
0709 12:15:40.097184 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.097312 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5570,"prepare.queue_time_us":148767,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":87,"raft.queue_time_us":12492,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":29060,"spinlock_wait_cycles":34709632,"thread_start_us":202771,"threads_started":2,"wal-append.queue_time_us":175112}
W20260709 12:15:40.159339 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59863.8us
I20260709 12:15:40.161724 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.189679 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:40.147001 29169 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.162545 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.190650 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.141311 29220 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:40.191867 24011 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:40.146454 29359 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.192422 29359 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.192485 29352 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.193359 29097 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:40.193941 29097 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:40.194963 29097 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.169900 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.169202 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.160847 29105 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.175261 23463 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:40.160634 29288 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:40.160775 29278 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:40.178687 29366 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.191902 29220 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.198323 29169 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.200646 29366 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.202171 29105 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.202641 29169 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.202759 29366 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.203655 29173 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.204049 29278 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.205977 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.165861 29208 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.208644 29379 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:40.210464 29359 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.214583 29345 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.216696 29345 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.216523 29377 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:40.218259 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.212790 29359 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.228746 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.202206 29220 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.231475 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.227764 29173 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:40.229058 29377 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:40.235121 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.236194 29359 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:40.217324 29367 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:15:40.247510 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2662) took 1281 ms. Trace:
I20260709 12:15:40.243290 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.232087 29344 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:40.247948 23469 rpcz_store.cc:276] 0709 12:15:38.966231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:38.967268 (+  1037us) service_pool.cc:224] Handling call
0709 12:15:38.967551 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:39.270547 (+302996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:39.270580 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:39.270622 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:39.270653 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.247276 (+976623us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b540044af54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:40.247328 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.247455 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696379776}
I20260709 12:15:40.233661 29278 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:15:40.241721 29323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.006s	sys 0.008s
I20260709 12:15:40.249269 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.249624 29025 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:40.236879 29173 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.262475 29208 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.265766 29173 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:40.252996 29323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.006s	sys 0.009s
I20260709 12:15:40.275364 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.274436 29368 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:40.250959 28875 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:40.275687 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.277714 29288 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:40.277410 29206 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.250924 29352 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.306301 29193 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:40.283355 29319 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.265254 29322 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.292403 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.294590 29343 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:40.307104 29352 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.251448 29159 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:40.310079 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:40.311280 29208 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.314217 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1565) took 1159 ms. Trace:
I20260709 12:15:40.317240 29175 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.337046 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.320335 29206 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.338177 29352 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.338294 29175 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.339824 29322 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:40.339912 29206 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.340301 29322 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.323174 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.340756 29322 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:40.341696 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.342772 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.343726 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.344210 23462 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:15:40.345857 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.346117 29146 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.322312 29382 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.324095 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.334406 29319 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.347070 29382 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.347474 23465 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:40.349098 29319 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.324802 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.352296 23461 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:40.325248 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.328578 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.304543 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.353091 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:40.290482 28875 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:40.349978 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.354122 24008 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:40.353062 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.354661 23313 raft_consensus.cc:2368] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:40.353255 23461 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:40.338188 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.355850 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.356307 23321 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:40.337049 23316 rpcz_store.cc:276] 0709 12:15:39.154970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.155145 (+   175us) service_pool.cc:224] Handling call
0709 12:15:39.155447 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:39.188338 (+ 32891us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:39.416913 (+228575us) log.cc:844] Serialized 14 byte log entry
0709 12:15:39.571766 (+154853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:39.571837 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:39.571897 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:40.313966 (+742069us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b5400295754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:40.314021 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:40.314158 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18698,"prepare.queue_time_us":137930,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":1709,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":28363,"spinlock_wait_cycles":75072384,"thread_start_us":374435,"threads_started":2,"wal-append.queue_time_us":253422}
I20260709 12:15:40.359107 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.360616 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.361193 23318 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:40.357267 23320 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:40.362186 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.364462 29323 raft_consensus.cc:2764] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:15:40.364944 29383 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:15:40.362404 29285 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:40.365573 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.365744 29285 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.350886 29165 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:15:40.365006 24015 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 12:15:40.369776 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.368064 29285 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.370270 23321 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:40.351344 29208 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.371815 23464 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:40.330327 29193 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:40.371987 23394 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 10. Current term is 11. Ops: []
I20260709 12:15:40.375690 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.378046 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.378623 23394 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:40.381990 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:40.336433 29343 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:40.349200 29382 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.387290 23462 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:40.399438 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.400408 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.400993 23394 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:40.385685 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.358542 29097 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.402848 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.364856 29144 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:40.405694 23394 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:40.406320 29169 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:15:40.407722 29370 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:40.373417 29206 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.375288 29319 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:40.418242 29288 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.418790 29288 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.417738 29105 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.422995 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.358016 28875 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.421191 29288 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.435087 23394 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:40.431751 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.411123 29352 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:40.435237 23389 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:40.408528 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.439055 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.389750 29323 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:15:40.413633 29169 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.442752 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.447078 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.444407 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.425704 29285 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.459100 29285 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.446306 23392 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:15:40.462307 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.462379 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:40.463037 23392 raft_consensus.cc:606] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:40.418264 29319 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.450350 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.464247 29319 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:40.466015 23395 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:40.450811 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:40.451942 23388 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:40.450690 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.445640 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:40.455227 29379 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:40.457257 29283 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:40.446157 29343 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.448850 29286 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:40.448985 29370 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:40.468982 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.416491 29144 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.453933 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.433897 29345 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.477532 29345 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.478802 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.480276 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.476537 29292 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:40.481338 29318 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:40.413326 29097 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.482858 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:40.449391 29169 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:15:40.483379 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.441759 29208 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.461277 29285 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.485874 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:40.505982 24008 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:40.487195 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.488134 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.510450 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.496357 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.474902 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.511547 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.448069 28875 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:40.517814 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.519574 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.429741 29206 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.509061 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.509800 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.480791 29345 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.430791 29105 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.436534 29326 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:40.529203 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.530078 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:40.525329 24008 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:40.496150 29318 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:40.436925 29193 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.476523 29322 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:40.547399 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.678741 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:40.811784 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.800726 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.574932 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:40.800516 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.542481 24009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:15:40.501842 28842 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:40.812681 23312 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:40.544134 24011 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:15:40.822867 24011 pending_rounds.cc:85] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 10
W20260709 12:15:40.809192 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 279209us
I20260709 12:15:40.823292 24011 pending_rounds.cc:107] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:15:40.823721 24011 raft_consensus.cc:2889] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:40.692817 23980 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.825202 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.813553 24009 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.817747 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.818347 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:40.818784 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.824441 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.827587 24009 raft_consensus.cc:614] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:15:40.828359 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 149759us
I20260709 12:15:40.819062 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.512501 29292 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:40.589908 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:40.714716 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 195910us
W20260709 12:15:40.824278 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.538942 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:40.824396 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:40.833281 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 294406us
I20260709 12:15:40.840759 23464 raft_consensus.cc:2410] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:15:40.533337 29343 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:40.884856 29322 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.885526 29193 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:40.885771 29389 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:40.886281 29110 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:40.886859 29110 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:40.887646 29286 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:40.888945 29110 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.533061 29288 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:40.633558 29387 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.324s	user 0.004s	sys 0.000s
I20260709 12:15:40.889717 29288 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:40.895380 29288 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:40.896564 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:40.891734 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:40.896996 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de7238eb389343cd954cb756bed5925f"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:40.897625 23396 raft_consensus.cc:606] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.898087 23396 raft_consensus.cc:614] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:40.556097 29220 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:40.734459 29396 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.079s	user 0.000s	sys 0.005s
W20260709 12:15:40.733345 29394 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
W20260709 12:15:40.745714 29397 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.087s	user 0.006s	sys 0.000s
W20260709 12:15:40.892918 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68557.6us
I20260709 12:15:40.894191 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:40.761946 29349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.000s	sys 0.018s
I20260709 12:15:40.577385 28972 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:40.899197 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 68937.5us
W20260709 12:15:40.923789 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.889997 29146 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:40.903250 29390 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:40.638906 29313 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:40.902446 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.586421 29097 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.687700 29368 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:40.568835 29105 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.629086 29025 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:40.633263 29338 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:40.657155 28997 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.607226 29334 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:40.607473 29283 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:40.940505 29283 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:40.944128 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.608281 29206 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.590896 29359 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:40.904055 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:40.950208 23396 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:40.596223 29175 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.708585 29169 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.824815 29323 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:40.906204 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:40.962261 23465 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:40.963464 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:40.964327 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:40.890506 29208 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:40.907133 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:40.966044 23464 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:15:40.574749 29144 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:40.687471 29366 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:40.909035 29279 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:15:40.828393 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 135688us
W20260709 12:15:40.916307 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.822247 29318 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:40.921983 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:40.922986 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:40.591032 29326 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.889851 28842 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:40.925594 23466 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:15:40.986291 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:40.900429 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.672962 29173 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.007797 29173 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:40.608560 29285 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.312s
W20260709 12:15:40.924594 29349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.001s	sys 0.029s
I20260709 12:15:40.904232 29404 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:40.572973 29319 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.012985 29202 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.013837 29169 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.014246 29202 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.014899 29319 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:40.935210 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 415634us
I20260709 12:15:40.921989 29322 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.015810 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:41.015743 29322 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.017460 29279 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:41.017622 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.017952 29322 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:40.903507 29286 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:15:41.018002 29279 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.018862 29286 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:41.019209 29286 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:41.020114 29279 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.020013 29286 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.020726 29402 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:41.020895 29202 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.021349 29286 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:40.889195 23215 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:40.987355 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 86980.6us
W20260709 12:15:40.988481 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65602.7us
I20260709 12:15:41.050380 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:40.951714 29338 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.021205 29402 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:40.953603 29334 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.993180 29345 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:40.996243 29288 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:40.981047 29366 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:40.954056 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.023597 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.027566 23312 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:41.054363 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:40.992385 29289 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:40.986035 29323 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:41.039316 23314 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:41.058576 23319 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:41.057613 23318 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:41.055436 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:41.056958 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:40.951539 28997 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.037715 28875 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.066780 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.053800 29402 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.071936 23434 leader_election.cc:400] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:41.072178 23980 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:41.067958 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.072682 23434 leader_election.cc:403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:15:41.069489 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 167678us
I20260709 12:15:41.022264 29266 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:15:41.135146 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.074139 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.074546 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:41.157032 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 82582.2us
W20260709 12:15:41.052202 24016 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.055s	sys 0.036s
I20260709 12:15:41.157828 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:41.158337 24016 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.056s	sys 0.036s
I20260709 12:15:41.055914 29173 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:41.133474 29349 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
W20260709 12:15:41.136883 29400 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:15:41.061676 29319 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:41.084841 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.071967 29289 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.056100 28842 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:41.067958 29382 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:41.163369 29175 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.160048 29206 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:15:41.071323 28875 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.072362 28964 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:41.170702 28997 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.179950 29193 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:41.188536 23215 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 11, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.185034 29289 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:41.174281 29366 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.194406 29366 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:41.197018 28875 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.190506 23215 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.179809 29382 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:41.198889 29382 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:41.201977 23215 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:41.203727 23215 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 13, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 13 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.210160 29338 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.172016 29389 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:41.137809 29370 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:41.138315 29025 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:41.177675 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 92962us
I20260709 12:15:41.212306 23468 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:41.212481 29025 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.211221 23215 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.213136 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.074093 29313 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:15:41.186092 29406 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:41.215787 29406 consensus_queue.cc:1243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:41.216246 29313 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:41.183251 29392 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:41.217036 29313 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:41.217756 29370 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:41.218391 29313 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:41.218770 29370 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:41.219174 29370 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:41.219267 29313 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:41.133872 29278 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:15:41.177837 29319 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.206221 29334 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.207288 29288 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:41.186750 28964 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:41.209100 29289 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:41.214696 29025 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.169423 29202 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:41.214560 28875 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:41.069203 29144 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 12:15:41.223978 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.185907 29345 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.257252 23387 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:41.219696 29313 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:41.259114 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 191245us
I20260709 12:15:41.223186 29146 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:41.261677 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 189571us
I20260709 12:15:41.256467 28964 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:41.258901 29202 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.271291 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:41.224452 29390 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:41.270115 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:41.259364 29292 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:41.277364 29322 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:41.277509 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:41.214169 23215 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.275830 28970 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.261802 29193 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:41.278311 29390 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.282231 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:41.278487 29278 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:41.293299 23466 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:41.290572 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.187721 29323 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.298532 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.300813 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.301163 23461 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:41.301517 23318 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:41.281651 29202 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:41.302323 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.261545 29353 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:15:41.302340 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.219944 29370 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.305398 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:41.299316 29313 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.306615 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.307344 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:41.276461 29289 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:15:41.309759 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:41.325807 23320 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.310528 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.286904 29315 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:15:41.311260 28972 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.311817 28842 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.312347 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:41.313221 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.314749 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:41.316145 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.318694 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:41.319866 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:41.327026 23313 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:41.330937 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.331604 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.328399 29279 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:41.332428 29279 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.332873 29289 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:41.334584 29279 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.334419 28972 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.337074 28972 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:41.337738 23315 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:41.339148 28842 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.343146 29097 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:41.343185 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.343873 28842 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.344285 28972 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:41.331851 29313 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:41.345813 29326 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:15:41.361886 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.382726 23314 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:15:41.380062 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 305966us
I20260709 12:15:41.384263 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:41.304091 29343 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:41.386796 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.389081 28842 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:41.391049 28842 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.382521 28972 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:41.391630 23316 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:41.394158 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2818) took 1547 ms. Trace:
I20260709 12:15:41.321120 29353 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:41.394699 23387 rpcz_store.cc:276] 0709 12:15:39.846561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:39.846785 (+   224us) service_pool.cc:224] Handling call
0709 12:15:39.864699 (+ 17914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:41.269412 (+1404713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.393859 (+124447us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000a7b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:41.394083 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3202099968}
I20260709 12:15:41.292685 28970 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.303913 29401 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:41.404273 28842 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.404608 29334 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.461020 29401 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.461771 29334 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.463177 29401 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.409581 28972 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.424602 29288 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:15:41.426049 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.428287 29399 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:41.349700 29146 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.360519 29317 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.439632 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.440866 29206 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:41.371068 29169 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:41.452466 29319 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.392994 29370 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:41.456110 29382 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.457947 23215 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:41.349293 29318 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.406379 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.566265 23313 raft_consensus.cc:2410] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
W20260709 12:15:41.464251 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 77539.2us
I20260709 12:15:41.465317 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.467186 23320 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:41.382892 29097 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:41.385121 29326 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:41.572809 29097 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.574448 29097 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:41.574262 29326 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.575842 29326 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:41.527036 29315 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.484999 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 171418us
W20260709 12:15:41.489959 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:41.469831 29220 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.001s
I20260709 12:15:41.468001 29292 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:41.556767 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.589970 23321 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:41.469662 29288 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:41.570858 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.571125 28964 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:41.484221 23436 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.560s (SENT) [suppressed 8 similar messages]
W20260709 12:15:41.559871 29286 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 12:15:41.532728 29105 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:41.586766 23465 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:41.555910 29334 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.604349 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:41.623144 24011 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:41.605357 24008 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:41.623957 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.605747 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:41.606383 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.607250 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.608516 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.480760 29402 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.628818 24008 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:41.629158 29421 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.605914 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:41.630661 29025 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:41.631170 29025 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:41.631414 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:41.632671 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.632185 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.560177 29169 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.554227 29206 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.652108 24012 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:41.652910 23317 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:41.547806 29317 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.641534 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.644676 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.645852 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.646571 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.646662 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.648175 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.648373 23396 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:41.593057 29220 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.635320 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.667759 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.668813 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:41.602671 29366 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.663118 23393 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:15:41.664664 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.651844 29208 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:41.630036 29402 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:41.679921 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:41.632654 29411 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.648953 29025 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:41.588449 29315 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.622869 29414 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:41.650758 29323 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.691104 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.692500 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:41.706588 23388 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:41.710366 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.629791 29421 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.712864 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.681109 29286 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:41.716224 24014 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:41.724285 23388 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:41.724658 24014 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:41.716995 24012 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:41.718612 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 234425us
I20260709 12:15:41.720616 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.720813 23316 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:41.687012 29343 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:15:41.726657 23468 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:41.727339 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.728950 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.705335 29366 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:41.729679 24012 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:15:41.732945 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:41.709621 29431 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.739897 24012 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:41.745744 29431 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.746277 23215 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 13, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 13 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.748315 29431 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.753669 29404 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:41.720034 29193 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:41.718091 29208 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:41.713544 29323 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.774780 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.694231 29288 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.741223 29419 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:41.776012 23468 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:15:41.788358 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:41.760705 29366 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:41.797231 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.735448 29220 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:15:41.701815 29411 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.748951 23215 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:41.829290 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:41.792095 29419 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.832614 29431 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:15:41.838606 29424 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:15:41.844616 24011 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:15:41.851557 29105 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:41.869962 29431 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:41.870337 29315 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.870558 29419 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:41.828928 29169 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.829362 29382 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.831058 29318 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:41.853106 29202 consensus_meta.cc:287] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.002s
I20260709 12:15:41.850775 29317 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.875850 29105 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:41.863567 28970 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.876065 29431 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:41.845522 29404 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 12:15:41.886925 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 218189us
I20260709 12:15:41.895418 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.898996 28970 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:41.806946 29366 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.906814 23468 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:41.886965 29202 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:41.914048 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:41.873411 29208 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.916790 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:41.931741 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:41.913115 29437 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:41.953820 29422 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 12:15:41.935488 29423 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:15:41.863829 23215 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 13 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:41.984979 24011 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 12:15:41.962677 29220 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:41.991482 23395 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 12:15:41.992249 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.992905 23393 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:41.986655 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:41.993693 23389 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:41.986660 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:41.985072 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.964344 29323 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:41.995287 23388 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:41.997545 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:41.964125 29437 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.000349 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.000938 24008 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:41.991542 29220 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.006317 29220 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:41.981014 29382 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.008209 29382 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:41.993546 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:41.966951 29334 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.010354 29334 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.011250 29382 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.012571 29334 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:41.987248 29427 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:15:42.018956 23389 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:42.022239 29345 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.026281 29345 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.025889 23215 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.028357 29345 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.024756 29202 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.963363 29441 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:15:42.031180 28842 consensus_meta.cc:287] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.003s
I20260709 12:15:42.031597 28842 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.025309 29220 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.027678 23215 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 14, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 14 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:42.032687 24013 raft_consensus.cc:2368] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14.
I20260709 12:15:42.033959 29366 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.033354 23215 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:42.035771 23215 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 15 to 17, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:42.042364 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1585) took 1467 ms. Trace:
I20260709 12:15:42.043015 29220 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.042745 23321 rpcz_store.cc:276] 0709 12:15:40.574859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.574998 (+   139us) service_pool.cc:224] Handling call
0709 12:15:40.575280 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:41.590347 (+1015067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.042118 (+451771us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003af054. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:42.042305 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333162368}
I20260709 12:15:42.023466 24010 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
W20260709 12:15:42.065636 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 299478us
I20260709 12:15:41.998040 29411 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:41.933480 29025 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.067734 29370 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.001363 29437 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.075428 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.078431 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:42.078912 29447 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:42.081666 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:42.081775 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.086277 23316 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:42.084748 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.099890 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.100493 23389 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:42.102385 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.103293 24009 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:42.079582 29447 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:42.106916 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:42.085989 29323 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.110378 23470 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
W20260709 12:15:42.109251 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 126643us
I20260709 12:15:42.079993 29370 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.123536 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.124205 23316 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:42.120838 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.121091 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.105021 28970 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:15:42.125578 23463 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:42.122092 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.086301 29366 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:42.123226 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.091801 29412 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:42.128132 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.131618 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.132509 23467 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:42.130066 23465 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:42.129395 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.108582 29318 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:42.130534 23463 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:42.091897 29441 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:15:42.129294 28970 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:42.134224 29353 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:42.136898 23464 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:42.140230 29449 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:15:42.124372 29315 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:42.144886 23469 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:42.135478 29318 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.154321 24009 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:42.150585 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.157409 29442 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:42.168463 29440 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:42.168844 29440 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:42.169742 29440 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.171048 29440 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:42.151979 29450 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:15:42.174808 29450 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:42.175151 29450 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:42.176326 29417 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.553s	user 0.001s	sys 0.001s
I20260709 12:15:42.175989 29450 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.177325 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1328) took 1707 ms. Trace:
I20260709 12:15:42.177639 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.178354 23465 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:42.143224 29353 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.161149 29370 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.161687 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.165540 29441 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.192569 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.179615 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.193136 23467 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:15:42.193603 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:42.160121 29025 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.127864 29323 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.177345 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.177726 24016 rpcz_store.cc:276] 0709 12:15:40.469572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:40.469888 (+   316us) service_pool.cc:224] Handling call
0709 12:15:40.497909 (+ 28021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.158833 (+660924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.158879 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.217646 (+ 58767us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:41.218045 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 12:15:41.525837 (+307792us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:41.525902 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.580270 (+ 54368us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b540008e8d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:41.580345 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.175782 (+595437us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:42.177101 (+  1319us) raft_consensus.cc:1685] finished
0709 12:15:42.177125 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.177269 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39294,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":132319616,"thread_start_us":931657,"threads_started":2,"wal-append.queue_time_us":351868}
I20260709 12:15:42.196571 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.148283 29315 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.156087 29105 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.188405 28842 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.189851 29431 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.184760 29353 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:42.194669 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.195726 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.196645 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.197288 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.221588 23321 raft_consensus.cc:2410] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:15:42.221781 23313 raft_consensus.cc:3060] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:42.217074 29421 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.198258 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.143339 29169 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:15:42.156548 29318 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:42.200259 23463 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:42.234465 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.191227 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.236346 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:42.236651 23466 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:42.202883 29418 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.482s	user 0.002s	sys 0.000s
I20260709 12:15:42.238783 29220 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:42.239171 29220 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.239543 29220 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:42.197098 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.207819 29370 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.182281 29450 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:42.209946 24011 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:42.194605 23465 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:42.207554 29382 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:42.212095 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.195106 29208 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:42.212649 29105 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.216320 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.244926 23982 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.228526 23464 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:42.252856 23314 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:42.200559 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.201946 29025 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:42.208604 29315 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:42.240787 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43759.2us
I20260709 12:15:42.255813 23468 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:42.203725 29411 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.242053 24009 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:42.209549 29206 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.245551 23467 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:42.247599 23313 raft_consensus.cc:2468] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:42.186566 29443 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:42.260476 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.205750 29447 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.218946 29412 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:15:42.267536 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.292754 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:42.298740 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.306510 24008 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:42.298238 29399 raft_consensus.cc:2804] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:42.307996 23314 raft_consensus.cc:2468] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:42.308462 29411 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.309702 28842 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:42.310133 28842 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.310485 28842 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:42.310720 29411 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:42.314738 29418 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:15:42.315887 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1594) took 1042 ms. Trace:
I20260709 12:15:42.316255 23312 rpcz_store.cc:276] 0709 12:15:41.273547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.273766 (+   219us) service_pool.cc:224] Handling call
0709 12:15:41.274167 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.279043 (+  4876us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:41.279412 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.613893 (+334481us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.789728 (+175835us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.793566 (+  3838us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.793869 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.793989 (+   120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:41.794563 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:15:41.794732 (+   169us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:41.794780 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:41.794873 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.202381 (+407508us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:42.315683 (+113302us) raft_consensus.cc:1685] finished
0709 12:15:42.315705 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.315832 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5231,"prepare.queue_time_us":44121,"prepare.run_cpu_time_us":165,"prepare.run_wall_time_us":168,"raft.queue_time_us":64224,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":18175,"spinlock_wait_cycles":19902080,"thread_start_us":467533,"threads_started":3,"wal-append.queue_time_us":388458}
I20260709 12:15:42.248520 29370 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.320429 29370 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.321334 29399 raft_consensus.cc:697] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:42.321926 29323 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.322315 29399 consensus_queue.cc:237] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:42.324096 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56657.8us
I20260709 12:15:42.326323 23980 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.257725 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:42.260442 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.258198 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:42.248517 23320 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:42.260989 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.259871 29382 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:42.402864 29432 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:15:42.382581 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54917.3us
I20260709 12:15:42.383412 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.404417 23314 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:15:42.252494 29208 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.244216 29421 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:42.406641 23982 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.288195 29431 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.390861 29455 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.395018 29458 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:42.396466 29202 raft_consensus.cc:2804] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:42.396603 23321 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:42.407939 29421 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.411868 29421 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:42.413102 29202 raft_consensus.cc:697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:42.388336 23466 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:42.207955 29441 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.400367 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.329077 29025 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.400924 23389 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:42.422506 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.407672 23320 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:42.418807 23467 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:42.424021 29438 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:42.425802 23321 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:15:42.400595 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 142505us
I20260709 12:15:42.408041 29208 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:42.410998 29455 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.430161 23316 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:15:42.303417 29419 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.430964 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.408175 29412 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.431757 23466 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:42.386900 29443 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.424572 29438 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:42.433465 23467 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:42.435041 23982 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.435810 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.439270 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.439747 24009 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:42.441347 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.441670 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.443338 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.439743 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.457535 24009 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 12:15:42.431559 29431 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:42.437106 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:42.414043 29202 consensus_queue.cc:237] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.401898 29382 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.553206 24016 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:15:42.427608 29025 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.575930 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.569885 29390 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:42.579625 24011 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:42.552695 29323 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.480265 29411 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:42.581197 23315 raft_consensus.cc:2410] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:15:42.582026 24012 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:42.583110 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 143907us
I20260709 12:15:42.450086 29412 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.554891 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:42.583904 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 142623us
I20260709 12:15:42.574272 29370 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.631390 29370 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.632104 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:42.589254 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 166793us
W20260709 12:15:42.635849 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 81059.6us
I20260709 12:15:42.576704 29392 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:42.575683 28875 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.577015 29457 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:42.595010 29431 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:42.617209 29442 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:42.645012 29390 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.648931 29146 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:42.674594 29412 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.659590 28875 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.675446 28964 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.677153 28875 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:42.677958 29428 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.050s	user 0.003s	sys 0.000s
I20260709 12:15:42.678399 29370 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.686311 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1244) took 1081 ms. Trace:
I20260709 12:15:42.686867 24013 rpcz_store.cc:276] 0709 12:15:41.604751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.623570 (+ 18819us) service_pool.cc:224] Handling call
0709 12:15:42.686044 (+1062474us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54003cb754. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:42.686229 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904220544}
W20260709 12:15:42.697463 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60026.1us
I20260709 12:15:42.640666 29315 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.662807 29442 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.701383 23321 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:42.689767 29146 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.602118 29438 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.647730 29323 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.668975 29432 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:15:42.725045 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.566s	user 0.009s	sys 0.015s
I20260709 12:15:42.653872 29392 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.714272 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.660365 29440 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:15:42.691910 29455 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.726011 23320 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
W20260709 12:15:42.726852 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 289843us
I20260709 12:15:42.715797 29442 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:42.718569 29146 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:15:42.747689 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 115635us
W20260709 12:15:42.733376 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.575s	user 0.009s	sys 0.015s
W20260709 12:15:42.752556 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 1.59475e+06us
I20260709 12:15:42.754240 24013 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:42.752398 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53760.2us
I20260709 12:15:42.760499 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2094) took 1459 ms. Trace:
I20260709 12:15:42.761675 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.762606 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.763262 23314 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
W20260709 12:15:42.763779 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.760941 23319 rpcz_store.cc:276] 0709 12:15:41.300732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.300838 (+   106us) service_pool.cc:224] Handling call
0709 12:15:41.306722 (+  5884us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:41.464150 (+157428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.464208 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:41.464778 (+   570us) log.cc:844] Serialized 52 byte log entry
0709 12:15:41.653737 (+188959us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:41.701531 (+ 47794us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.716770 (+ 15239us) log.cc:844] Serialized 14 byte log entry
0709 12:15:41.729366 (+ 12596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.731129 (+1001763us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:42.731182 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:42.760294 (+ 29112us) raft_consensus.cc:1685] finished
0709 12:15:42.760317 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:42.760447 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29892,"prepare.queue_time_us":76731,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":94955,"raft.run_cpu_time_us":587,"raft.run_wall_time_us":35933,"spinlock_wait_cycles":354753792,"thread_start_us":94575,"threads_started":1,"wal-append.queue_time_us":207111}
I20260709 12:15:42.770176 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.771047 24012 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:42.727471 29432 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.723196 29441 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:42.779577 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.782501 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.782531 23982 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:42.786405 23394 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:42.783010 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.731560 29440 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:42.784350 24009 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:42.780277 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.790809 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.790604 24012 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:42.795624 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.735498 29323 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.791414 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:42.797406 23394 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:42.801528 29443 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.798008 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.803843 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2697) took 1160 ms. Trace:
I20260709 12:15:42.804255 29390 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.804514 29443 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:42.805037 29443 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.805410 29443 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:42.807611 29466 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.754573 29419 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.753888 29411 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:42.814532 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.814424 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.815506 23469 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:42.804432 23470 rpcz_store.cc:276] 0709 12:15:41.643430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.645536 (+  2106us) service_pool.cc:224] Handling call
0709 12:15:41.649144 (+  3608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.122388 (+473244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:42.803619 (+681231us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54002a0dd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:42.803785 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1053634304}
I20260709 12:15:42.817294 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:42.817595 23314 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:42.817454 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:42.783563 29432 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:42.817927 23321 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:15:42.818521 23469 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:42.818467 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.821693 29457 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:42.823035 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:42.823382 29433 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:42.823911 29433 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.824287 29433 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:42.825735 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:42.801581 29353 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.005s
I20260709 12:15:42.831740 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.750003 29468 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.084s
W20260709 12:15:42.815506 29318 consensus_meta.cc:287] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.573s	user 0.016s	sys 0.001s
I20260709 12:15:42.731071 29431 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.832424 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.840391 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.845965 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:42.846602 23388 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:42.809373 29470 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:42.831600 29278 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:42.860191 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:42.861471 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 108144us
I20260709 12:15:42.810665 29441 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:15:42.863941 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:42.865381 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:42.836689 29025 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:42.808177 29466 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.853961 29317 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:42.855387 29476 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:42.890102 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:42.889729 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:42.857622 29470 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.905759 29475 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:42.907378 23388 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:15:42.908332 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 148919us
I20260709 12:15:42.890798 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:42.860754 29315 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.913672 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:42.867285 29370 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:42.871095 29440 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:42.846207 29318 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.894140 29470 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:42.837136 29353 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.891904 29317 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:42.924453 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52892.9us
I20260709 12:15:42.893218 29323 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:42.925618 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55911.2us
I20260709 12:15:42.926355 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:42.915228 28964 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:42.917949 29370 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:42.930670 23317 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:42.907261 29025 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:15:42.934439 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:42.839627 29457 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:42.929944 29450 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.677s	user 0.000s	sys 0.010s
I20260709 12:15:42.925064 29441 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:42.926887 29317 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:42.932220 28842 consensus_meta.cc:287] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 12:15:42.927557 29479 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:42.931774 29468 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:42.934933 29370 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:42.940613 29472 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:42.948259 28842 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.953780 29472 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.954401 29472 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:42.955632 28842 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:42.958698 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:42.958907 29442 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.959383 23470 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:42.950107 29479 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:42.961658 29479 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:42.965803 29441 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:42.966060 29318 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.966504 29353 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.966382 29441 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.968458 29441 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.944780 29484 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:15:42.945241 29392 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:42.890489 29411 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:42.924458 29208 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.936062 28964 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:42.945749 29429 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.315s	user 0.008s	sys 0.000s
W20260709 12:15:42.991482 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53926 (request call id 1246) took 1386 ms (client timeout 1804 ms). Trace:
I20260709 12:15:42.936990 29482 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 12:15:42.946695 29437 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:42.912644 29476 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:42.950485 29421 consensus_meta.cc:287] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.538s	user 0.003s	sys 0.008s
I20260709 12:15:42.977582 23470 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:42.989207 29278 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:43.000633 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:43.001098 29466 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.001214 29278 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:43.002081 29477 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:43.002746 29208 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:43.003309 29483 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:15:43.003580 28964 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:42.977671 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:42.988379 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74829.7us
I20260709 12:15:42.988907 23462 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
W20260709 12:15:42.992057 24010 rpcz_store.cc:269] 0709 12:15:41.605282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.623663 (+ 18381us) service_pool.cc:224] Handling call
0709 12:15:42.991277 (+1367614us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54005613d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:42.991404 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":916926208}
I20260709 12:15:43.005954 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.004750 23467 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:43.007411 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.008145 23469 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:43.011852 29359 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:43.016007 29359 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:43.016290 29476 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.016937 29359 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.018067 29359 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.017999 29432 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.010223 29441 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:42.997851 29437 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:43.030426 29437 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.034881 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:43.041987 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 152070us
I20260709 12:15:43.043578 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.006278 29421 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.046207 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.047911 29421 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.047956 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.048848 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.048794 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.049582 24015 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.049697 24009 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:43.049837 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.051923 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.051967 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.052609 24009 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:43.052731 23387 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:43.055181 29220 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.004s
I20260709 12:15:43.055603 29220 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.056743 24011 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:43.058298 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:43.048740 23388 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.058928 23313 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.058784 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.065651 24009 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:43.066365 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:43.045209 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.068277 23318 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.071177 23464 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:43.071812 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.068593 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:43.012661 29466 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:43.083949 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:43.084705 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 162503us
I20260709 12:15:43.084940 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.073339 23388 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:43.080024 24015 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:43.081935 23313 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:43.086547 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:43.086601 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.012902 29442 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.087342 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.091233 23470 raft_consensus.cc:2393] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:43.091539 23320 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 9: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.087164 23461 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:43.084028 29353 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:43.093179 29353 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.091410 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:43.103802 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.103933 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:43.105943 23387 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:43.038195 29437 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:43.105628 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:43.121837 23464 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:43.117762 29144 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:43.053048 29433 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.086910 29484 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:43.090315 29322 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:43.087141 29487 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:43.111986 28875 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:43.043836 29443 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.104534 29466 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:43.114516 29315 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:43.138298 29432 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.184659 29370 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:43.154142 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:43.154774 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.126413 29220 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.127342 29421 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:15:43.128536 29441 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.139689 29322 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.170778 29440 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:43.119669 29025 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:43.168859 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.192726 23320 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:15:43.193106 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.167953 29144 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:43.185058 29317 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.185992 28875 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.161048 29488 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:15:43.203253 29322 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:43.197860 29315 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.132172 29353 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.298247 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.207062 29470 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.270521 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1259) took 1064 ms. Trace:
I20260709 12:15:43.270401 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.200307 29146 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.203328 29476 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:43.262305 23314 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.205025 29472 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.224807 29432 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.230863 29025 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:43.203105 29441 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.276569 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.276882 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.358970 23394 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:15:43.274967 28875 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.282063 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.363029 23388 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:43.290560 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1609) took 1094 ms. Trace:
I20260709 12:15:43.290951 29317 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.201805 29479 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:43.294792 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 191068us
I20260709 12:15:43.302350 23393 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:15:43.303148 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.363852 23315 rpcz_store.cc:276] 0709 12:15:42.196238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.196364 (+   126us) service_pool.cc:224] Handling call
0709 12:15:43.290287 (+1093923us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b5400275bd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:43.290455 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874612096}
I20260709 12:15:43.220650 29487 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:43.305015 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.316751 23387 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.318392 23468 rpcz_store.cc:276] 0709 12:15:42.206310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.206469 (+   159us) service_pool.cc:224] Handling call
0709 12:15:42.206750 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:42.259323 (+ 52573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.270168 (+1010845us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54001063d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:43.270442 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112375680}
I20260709 12:15:43.273309 29488 consensus_queue.cc:1243] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:43.337687 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.361025 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.304447 29289 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:43.381373 23394 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 21: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:43.384321 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.384601 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.385190 24015 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.272267 29498 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 12:15:43.359609 29495 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:43.388132 29497 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:15:43.389729 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.363999 28972 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:43.338617 29392 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.402626 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1255) took 1177 ms. Trace:
I20260709 12:15:43.402983 24016 rpcz_store.cc:276] 0709 12:15:42.224816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.224965 (+   149us) service_pool.cc:224] Handling call
0709 12:15:42.238265 (+ 13300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:42.578525 (+340260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.401710 (+823185us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54007f0154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:43.402569 (+   859us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":617685888}
W20260709 12:15:43.399111 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 189516us
I20260709 12:15:43.398679 24015 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:43.400826 23464 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.271001 29466 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.358001 29432 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.339080 29476 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.381117 29144 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:43.395962 29443 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:43.398569 29488 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:43.393245 29433 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.386418 29289 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.422384 29289 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.422618 29146 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.408717 29392 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.412813 29488 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:43.423991 29488 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.424722 29392 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.426357 29441 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.426390 29434 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:43.427031 29434 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.428018 29470 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.392767 29315 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.414395 29370 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:43.435175 29496 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.303s	user 0.002s	sys 0.000s
I20260709 12:15:43.419090 29366 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:43.399102 29495 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.427879 29434 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.437868 29366 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.450004 29495 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:43.451861 29494 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 12:15:43.451987 29472 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:43.454231 29504 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:43.454799 29504 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:43.457578 29476 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:43.460930 29432 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.445031 29488 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:43.467121 29491 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.419s	user 0.006s	sys 0.000s
I20260709 12:15:43.455874 29504 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.468976 29504 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:43.469512 29479 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:43.470278 29492 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.245s	user 0.000s	sys 0.003s
I20260709 12:15:43.471525 29366 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.472885 29366 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:43.476191 29487 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:43.527570 23393 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
W20260709 12:15:43.532343 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:43.534576 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 415691us
I20260709 12:15:43.536620 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.542271 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.543641 23388 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:43.550984 23320 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:15:43.551537 23320 pending_rounds.cc:85] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:15:43.551862 23320 pending_rounds.cc:107] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:15:43.552255 23320 raft_consensus.cc:2889] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:43.554495 23321 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:43.555117 23315 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:43.557348 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.558298 23468 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.563730 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:43.564744 23312 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.565454 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.566128 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:43.566342 23466 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.567349 23317 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:43.572317 23468 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:43.578562 23315 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
I20260709 12:15:43.581485 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.581946 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.586056 23468 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:43.588443 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:43.589277 23318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43746 (request call id 1613) took 1296 ms (client timeout 1530 ms). Trace:
I20260709 12:15:43.589538 23320 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:15:43.589982 23318 rpcz_store.cc:269] 0709 12:15:42.292286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.292502 (+   216us) service_pool.cc:224] Handling call
0709 12:15:43.588606 (+1296104us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400148a54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:43.588941 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753117440}
I20260709 12:15:43.592120 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.593050 23394 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:43.597226 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.598973 23390 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.600626 23466 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:43.575304 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.606727 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2701) took 1169 ms. Trace:
I20260709 12:15:43.412681 29144 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.611456 29144 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.612571 29144 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:15:43.614339 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.414229 29025 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.615936 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.618053 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.618781 24009 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 12:15:43.619582 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.620347 24010 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:43.622274 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.623165 24013 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:43.626952 24015 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:43.607241 23467 rpcz_store.cc:276] 0709 12:15:42.437546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:42.507929 (+ 70383us) service_pool.cc:224] Handling call
0709 12:15:42.552462 (+ 44533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:43.007923 (+455461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.606035 (+598112us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54001077d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:43.606464 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039747840}
I20260709 12:15:43.628939 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.631409 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.632754 29318 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.633666 29144 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:43.634191 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.635006 24016 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:43.634514 29144 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:43.638674 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.640129 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:43.641772 29144 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.643043 29370 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.643765 29370 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.646903 29370 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.620493 24009 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:43.617108 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.606748 23313 raft_consensus.cc:2393] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.659682 23390 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:43.661859 23388 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:15:43.662384 23317 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:43.654857 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:43.675707 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:43.680534 23312 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
W20260709 12:15:43.687846 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.690423 29479 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:43.691943 29479 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:43.692436 29479 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:43.704361 24013 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:43.672492 29370 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:43.694736 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.709894 24014 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:43.710942 29334 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.711591 29334 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.652947 29318 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.717705 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:43.707597 29370 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.723421 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:43.728972 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 87691.4us
I20260709 12:15:43.737707 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.740396 29488 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:43.741544 29488 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.745797 23464 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
W20260709 12:15:43.749248 24014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1341) took 2024 ms (client timeout 1635 ms). Trace:
W20260709 12:15:43.750898 24014 rpcz_store.cc:269] 0709 12:15:41.724487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.727087 (+  2600us) service_pool.cc:224] Handling call
0709 12:15:43.748899 (+2021812us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b540074dbd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:43.749104 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4557246336}
I20260709 12:15:43.753916 29442 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:43.754889 29442 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:43.758852 29144 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.759577 29144 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.764199 29443 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:43.765000 29443 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.767444 29434 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:43.767637 29443 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:43.773324 29288 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:15:43.776196 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.777446 24011 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:43.779028 29464 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.779768 29464 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.780510 29144 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.785053 29278 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:43.788167 29441 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.790541 23387 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:43.790145 29441 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.789951 29479 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.792793 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.794045 23312 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:15:43.794423 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.794994 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.794334 29479 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.797957 29318 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.799127 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:43.800238 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.801397 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.802222 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:43.802575 24014 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:43.803150 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.726662 29370 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:43.805706 23312 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:43.789249 29334 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:43.745108 29488 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.805511 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.823097 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.804420 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:43.826478 24009 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 21: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:43.828819 23317 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:43.829288 24016 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:15:43.828284 23465 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:43.785460 29464 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.832172 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.836560 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.837731 29334 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:43.838219 29334 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:43.838650 29334 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:43.841275 29519 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:15:43.841395 29435 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:43.846698 29441 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.847467 29097 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:43.836258 29144 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:43.848143 29097 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.850658 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:43.851126 29097 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:43.851693 24012 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:43.852712 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.853694 29476 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:43.853938 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.808749 29442 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.854404 29476 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.856019 29462 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:43.856668 29462 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:43.857810 29462 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:43.858999 23467 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:15:43.859781 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.860636 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:43.861069 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:43.865128 23313 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:43.866392 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.866585 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.867213 23393 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:43.854405 23388 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:43.853144 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51138.5us
I20260709 12:15:43.872107 23467 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:43.872062 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.873167 23316 raft_consensus.cc:1240] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:15:43.873416 23470 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:15:43.850792 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.851735 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:43.878504 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50331.2us
I20260709 12:15:43.890923 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:43.892004 23464 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:43.893424 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:43.900317 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 97264.5us
I20260709 12:15:43.903555 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.904814 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:43.907444 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:43.908706 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.886801 29144 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.910451 23318 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:15:43.912138 23318 raft_consensus.cc:1240] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:15:43.912673 29144 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.903831 23316 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:15:43.913412 23318 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:43.913717 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:43.916227 23388 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:43.898229 24008 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
W20260709 12:15:43.913705 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 80112.3us
I20260709 12:15:43.923498 29399 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.924425 29435 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:43.925571 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 982) took 2280 ms. Trace:
I20260709 12:15:43.925375 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:43.925983 24007 rpcz_store.cc:276] 0709 12:15:41.606623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:41.607062 (+   439us) service_pool.cc:224] Handling call
0709 12:15:41.607476 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:41.640193 (+ 32717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:41.640219 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:42.706438 (+1066219us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b180044e3c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65552a6 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a
0709 12:15:43.694272 (+987834us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:43.714683 (+ 20411us) log.cc:844] Serialized 26 byte log entry
0709 12:15:43.827851 (+113168us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:15:43.831371 (+  3520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:43.837127 (+  5756us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:43.878653 (+ 41526us) raft_consensus.cc:1685] finished
0709 12:15:43.878672 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:43.887206 (+  8534us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21958,"raft.queue_time_us":41090,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":15856,"spinlock_wait_cycles":27415552,"thread_start_us":119139,"threads_started":2,"wal-append.queue_time_us":146148}
I20260709 12:15:43.930323 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:43.941452 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:43.942999 29220 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:43.943922 29220 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:43.944727 29220 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:43.945113 29220 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:43.945216 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.947508 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.948644 23396 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:43.948868 23314 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:43.947678 23319 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 12:15:43.953231 29476 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:43.954463 29322 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.650s	user 0.014s	sys 0.004s
I20260709 12:15:43.955114 23387 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:43.954950 29322 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:43.956293 23468 raft_consensus.cc:1275] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:43.957044 29322 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:43.957962 23470 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:15:43.958776 23320 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:43.959213 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:43.959527 23312 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:43.925949 23462 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:43.982784 23391 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:15:43.983810 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.985910 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:43.984519 23387 raft_consensus.cc:2376] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20. Re-sending same reply.
I20260709 12:15:43.995098 28842 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:43.996536 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:43.999894 23464 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:15:44.002781 23321 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:15:44.003233 23321 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:15:44.003525 23321 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:15:44.003759 23321 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:43.997396 28842 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:15:44.010157 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.012997 29202 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:15:44.013722 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 111135us
I20260709 12:15:44.014562 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:44.018726 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.021836 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.022056 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.023465 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.024050 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.024792 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.025267 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.018095 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:43.967101 29366 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:44.026149 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.026793 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.026932 29419 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:44.034706 29419 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:44.023474 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.025398 23396 raft_consensus.cc:2376] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16. Re-sending same reply.
I20260709 12:15:44.039705 23391 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:43.984716 29370 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.014997 29220 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.052661 23390 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
W20260709 12:15:44.053599 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 206477us
I20260709 12:15:44.054790 23315 raft_consensus.cc:2368] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:15:44.028365 29278 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:15:44.055635 23388 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:15:44.055873 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59460.5us
I20260709 12:15:44.067973 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.085752 28970 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:44.086375 28970 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.086833 28970 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:44.089573 29202 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:44.090402 29202 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:44.092700 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:44.093497 23392 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:44.057541 29322 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:44.058060 29366 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:44.027747 29412 consensus_queue.cc:1048] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:15:44.091791 29202 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.103154 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:44.102197 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44210.5us
I20260709 12:15:44.105986 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:44.106918 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:44.107792 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:44.077484 29419 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:15:44.110249 23387 raft_consensus.cc:2410] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:15:44.110733 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:44.117962 24016 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:44.119343 23312 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:44.121963 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.058976 29318 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:15:44.125828 23391 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:15:44.127239 29470 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:15:44.127722 29470 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.128077 29470 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:44.130154 29315 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:44.103868 28842 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:15:44.131021 28842 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:44.139212 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:44.142107 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.157862 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.143158 29322 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:44.161250 29322 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.166380 29433 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:15:44.173834 28875 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:15:44.177955 29437 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:44.187656 29220 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.188920 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:44.189847 23465 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:15:44.189184 29370 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.198143 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.154935 29315 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:44.200935 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:44.201704 23313 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:44.202284 23313 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:44.201886 29315 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.202714 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:44.211474 23465 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:44.219456 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:44.228493 23392 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:15:44.229374 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 94534.5us
W20260709 12:15:44.235653 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.241674 29493 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.242362 29493 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.243710 24011 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 9.
I20260709 12:15:44.232724 29435 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:44.250417 29435 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:44.251663 29435 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:44.254189 29520 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:15:44.257496 23312 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 9
I20260709 12:15:44.257849 23312 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:15:44.258090 23312 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:44.227980 29433 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:44.270673 29433 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.272534 29493 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.236146 29366 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.298374 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.299099 23465 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:15:44.300897 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:44.305578 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 73715.9us
W20260709 12:15:44.324067 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:44.325188 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 187213us
I20260709 12:15:44.359488 29432 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.367460 29432 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.380288 23313 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:15:44.390244 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62881.8us
W20260709 12:15:44.392042 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68565.6us
W20260709 12:15:44.412384 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:44.418670 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.420641 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:44.448770 29334 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.005s
I20260709 12:15:44.442963 29288 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:15:44.494274 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 97725.3us
I20260709 12:15:44.440758 29462 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:44.502446 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 78564.2us
I20260709 12:15:44.495745 29288 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.519433 28972 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:44.520160 28972 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.537636 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.495987 29334 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.515997 29288 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:15:44.575706 29536 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:44.585103 29504 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:15:44.539139 29462 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.547459 29432 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.591590 29334 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.592473 24014 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:44.595227 29527 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:15:44.599071 29546 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:44.599571 29546 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:44.600560 29546 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.603561 29464 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:44.604261 29464 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:44.609546 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 90179.4us
I20260709 12:15:44.606238 28972 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.606793 29442 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:44.609925 29462 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:44.627286 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 110614us
W20260709 12:15:44.628588 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.571967 29470 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:44.639071 29529 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
W20260709 12:15:44.641968 29532 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:15:44.634874 29442 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.647303 29442 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.652648 23318 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:44.645488 28970 consensus_meta.cc:287] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.005s
W20260709 12:15:44.652588 29530 log.cc:927] Time spent T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:15:44.673774 29546 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:44.691752 23392 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: [15.10-18.11]
I20260709 12:15:44.693631 29462 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.693809 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:44.694267 29462 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.695004 29464 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:44.695042 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.696622 24010 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:44.701615 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.702404 29479 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:44.703190 29479 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.703405 24016 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:15:44.709558 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.709797 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.660933 28970 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.713624 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:44.706450 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:44.718737 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.719334 23388 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 19
W20260709 12:15:44.722935 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.725008 23387 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:15:44.715482 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:44.734460 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:44.734623 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:44.740713 23392 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:44.740710 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:44.741480 24016 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:44.715067 29462 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.753824 24008 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:44.754794 24011 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:15:44.757588 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 73352.9us
I20260709 12:15:44.761768 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.740162 29470 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.760824 29553 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:44.764267 29553 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:44.764621 29553 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:44.773746 29554 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.774276 29476 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:44.774470 29554 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.775355 29476 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:44.775601 29553 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.776120 29479 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.776939 29479 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:15:44.784756 29479 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:44.786259 29479 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.777470 28970 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:44.790972 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.777606 29553 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:44.781543 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 159018us
I20260709 12:15:44.792894 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:44.794803 23360 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.795404 29479 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:44.796748 29488 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:44.784021 29462 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:15:44.784444 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:44.798103 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.798597 23388 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:44.809202 24016 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:44.819782 23387 raft_consensus.cc:3055] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:44.820441 23387 raft_consensus.cc:740] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:44.827535 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:44.828213 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50331.8us
I20260709 12:15:44.829129 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.828234 23390 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:44.828529 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.832131 23392 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:44.822674 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.833597 23393 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:44.834703 23396 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:44.834923 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:44.835477 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44861.5us
I20260709 12:15:44.777192 29554 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.805311 29462 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:44.842600 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.813761 29144 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:44.840512 23315 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:44.850143 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.851156 23390 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:15:44.839246 29220 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 12:15:44.854561 29550 consensus_queue.cc:1059] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:15:44.855782 29313 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.856449 29313 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.858381 23464 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:44.860253 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.827294 29553 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:44.861254 23390 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:44.818663 29470 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:44.871557 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 101304us
I20260709 12:15:44.858246 23387 consensus_queue.cc:260] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.811496 29288 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.874280 23468 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:44.862249 29318 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:44.887863 23462 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:44.918102 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:44.888967 23319 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:44.886363 29554 consensus_queue.cc:1059] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:15:44.926393 23320 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:44.927273 29318 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.931171 29470 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.931368 29462 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.932399 29462 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:44.933838 29313 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.854586 29547 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:15:44.939181 29288 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.940635 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.918042 29442 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:44.942209 29442 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:44.943733 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:44.944571 23392 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:44.944943 29442 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:44.945801 23392 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:44.948655 23394 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:44.949147 23394 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:44.950199 23394 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:44.951444 29553 raft_consensus.cc:3055] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:44.951951 29553 raft_consensus.cc:740] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:44.954093 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.954939 23390 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:44.955633 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.956944 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:44.957235 23315 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:44.957624 24011 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:44.942245 23466 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:44.941372 23468 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:44.974013 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:44.974936 23393 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:44.976121 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:44.985862 23388 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:15:44.985997 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:44.986546 23464 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:15:44.989637 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 50951.5us
I20260709 12:15:44.991142 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.994504 29547 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:44.996068 29561 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:44.996554 29561 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:44.993367 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.998772 29144 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:44.985139 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:45.002786 24010 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 10: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:45.003597 24014 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:44.984426 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:45.006840 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.010484 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:45.011123 23396 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:45.011304 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 214971us
I20260709 12:15:45.005208 23321 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:15:45.007975 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 71716us
I20260709 12:15:44.880383 29441 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.017693 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:44.983104 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:44.888652 29488 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:45.024541 24008 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:15:45.025334 23360 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.027757 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:44.952967 29553 consensus_queue.cc:260] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.030267 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:45.057049 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.008531 28875 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
W20260709 12:15:45.050523 29560 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:15:45.084280 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 83914.7us
I20260709 12:15:45.047011 29488 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:45.102336 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:45.111588 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:45.113153 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:45.113989 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 101984us
I20260709 12:15:45.135233 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.112628 29569 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:15:45.137801 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46990.9us
I20260709 12:15:45.082279 29366 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:45.079905 29504 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:45.163836 23360 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:45.150972 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.145017 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:45.094782 29442 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.143038 29366 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.141489 29569 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.047276 29441 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:45.196267 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 52446.1us
I20260709 12:15:45.203997 23394 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:45.206548 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 71444.9us
W20260709 12:15:45.209383 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 128480us
I20260709 12:15:45.142032 29546 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:15:45.195380 29366 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:45.218911 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.205727 29479 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:45.214951 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.226897 23464 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:15:45.229257 23396 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:45.142131 29433 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:15:45.198750 29569 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:45.203794 28875 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:15:45.217941 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 190240us
I20260709 12:15:45.209467 29313 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:15:45.214905 29550 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:45.242853 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.245321 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:45.246307 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 183878us
I20260709 12:15:45.181274 29442 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.252696 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.237502 29479 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:45.244655 29546 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:45.241653 29565 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:45.248548 29572 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:45.251576 29547 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:45.302258 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49271.5us
I20260709 12:15:45.314385 23464 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:45.258299 29479 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:45.251065 29493 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:45.243937 29202 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.062s
W20260709 12:15:45.327911 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 75281us
I20260709 12:15:45.259262 29546 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:45.341986 23982 leader_election.cc:336] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.520s (SENT)
I20260709 12:15:45.342528 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.344929 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:45.350632 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:45.313477 29572 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:45.341223 29544 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:15:45.244175 29488 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.331535 28970 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:45.311800 29547 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:45.354394 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 139528us
I20260709 12:15:45.350692 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:45.339303 29581 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:45.359907 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 222468us
I20260709 12:15:45.360018 29318 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.361809 29437 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:45.352833 24014 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:45.359051 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 44677.2us
I20260709 12:15:45.346658 29571 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.369382 24011 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:45.366641 24016 raft_consensus.cc:2368] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:15:45.373397 24014 raft_consensus.cc:2410] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:45.375849 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:45.378401 23982 leader_election.cc:336] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.560s (SENT)
I20260709 12:15:45.378897 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.357605 29572 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:45.352772 29568 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:15:45.377552 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 128579us
I20260709 12:15:45.310302 29565 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.354300 29433 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.366839 28875 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:45.340834 29493 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.370041 29547 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:45.390870 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89296.9us
I20260709 12:15:45.400064 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.401086 23465 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:45.376263 29585 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:45.383597 29552 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:45.408980 23362 leader_election.cc:400] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:15:45.359927 29169 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.162s
I20260709 12:15:45.384037 29559 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:45.414760 23362 leader_election.cc:403] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:45.380887 28970 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:15:45.402637 29488 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:45.393975 29412 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:45.380720 29318 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:15:45.421502 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:45.520283 23469 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:15:45.422603 29566 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:15:45.423375 29589 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:15:45.522318 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 100685us
I20260709 12:15:45.409529 29433 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.427166 29313 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:45.427955 23394 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:45.428228 29565 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:45.392808 29546 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.436829 29442 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.412113 29586 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:45.371277 29581 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.530829 29546 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:45.532595 29581 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:45.541615 29578 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.194s	user 0.001s	sys 0.001s
W20260709 12:15:45.542462 29576 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 12:15:45.521449 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1286) took 1233 ms. Trace:
I20260709 12:15:45.430148 29441 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.392352 29571 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:45.555156 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 166355us
I20260709 12:15:45.521574 29552 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:45.547492 23470 rpcz_store.cc:276] 0709 12:15:44.288483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.288604 (+   121us) service_pool.cc:224] Handling call
0709 12:15:44.298454 (+  9850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:44.298750 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:44.311737 (+ 12987us) log.cc:844] Serialized 14 byte log entry
0709 12:15:44.497780 (+186043us) log.cc:844] Serialized 14 byte log entry
0709 12:15:44.527024 (+ 29244us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b180034ff00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6587522 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:44.688623 (+161599us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b180034ff00. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6501c0a 00007ff0afc41219 00007ff0a6589a40 00007ff0a65844e4 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:45.066398 (+377775us) log.cc:844] Serialized 14 byte log entry
0709 12:15:45.520752 (+454354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:45.520831 (+    79us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:45.520889 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:45.520963 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:45.521337 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181381,"prepare.queue_time_us":79432,"prepare.run_cpu_time_us":623,"prepare.run_wall_time_us":4923,"raft.queue_time_us":345464,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":22354,"spinlock_wait_cycles":28534400,"thread_start_us":955072,"threads_started":4,"wal-append.queue_time_us":270702}
W20260709 12:15:45.586123 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57607.9us
I20260709 12:15:45.593875 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.594597 23465 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.595052 23465 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:45.626562 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 255449us
W20260709 12:15:45.627358 29587 log.cc:927] Time spent T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:15:45.628327 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:45.419863 29559 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.630139 23387 raft_consensus.cc:3060] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:45.588392 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 179282us
I20260709 12:15:45.643559 23360 leader_election.cc:400] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
W20260709 12:15:45.648926 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 246057us
I20260709 12:15:45.555138 29585 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:15:45.626567 29591 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.652467 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.652649 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:45.653723 29584 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:45.661758 29598 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.662009 29202 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.662359 29598 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.662503 29202 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.664536 29202 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.664816 29598 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:45.666100 29593 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.666617 29593 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.667289 28970 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.667865 28970 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.668455 29433 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.668845 29593 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:45.669746 23463 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:45.662016 29493 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.670903 29599 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:15:45.671825 29599 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:45.672151 29599 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:15:45.673547 29574 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:15:45.630183 29550 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:15:45.647441 29565 raft_consensus.cc:2764] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:15:45.688382 23387 raft_consensus.cc:1275] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:45.685635 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.665283 29598 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.693547 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.694247 23389 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:45.697664 29493 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.702308 29493 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.703262 29571 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.704356 29493 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.672919 29599 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.690311 29366 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.706465 29599 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:45.707768 29366 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.709646 23387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
W20260709 12:15:45.710140 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 81946.6us
I20260709 12:15:45.710471 23387 raft_consensus.cc:606] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.583982 28875 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.666134 29412 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:15:45.676303 29591 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.717970 29366 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:45.745266 29591 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:45.718124 29552 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.725726 29585 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:15:45.712657 29598 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.738791 29601 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.756314 29601 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.758987 29601 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.762828 29598 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.762961 29561 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.764384 29561 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:45.766669 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.754474 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.767381 23387 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
W20260709 12:15:45.769840 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58195.5us
I20260709 12:15:45.770779 23436 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.772250 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:45.773622 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:45.773706 29591 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:45.774344 29597 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:45.775020 29597 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.778815 29607 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:45.783118 29470 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.749517 28970 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:45.747808 29366 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.774457 29591 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:45.784361 29462 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.851s	user 0.007s	sys 0.007s
I20260709 12:15:45.785990 29366 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.792943 29547 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.793553 29479 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.795147 29547 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.795543 29479 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:45.796247 23462 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:45.747220 29569 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.798219 29569 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.750138 29585 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:45.808684 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59899.5us
I20260709 12:15:45.724097 23469 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:45.777359 29597 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:45.808568 29585 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.815631 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.823223 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.825524 29598 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:45.833679 29482 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:45.834221 29482 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.834620 29482 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:45.839792 29599 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:45.844456 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.825346 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.848208 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:45.848814 23389 raft_consensus.cc:606] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.849195 23389 raft_consensus.cc:614] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:45.855463 23464 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:45.855954 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.857885 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.858206 24014 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:15:45.867017 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:45.768517 29433 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:45.885946 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.887188 23465 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:45.889299 23387 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:15:45.890508 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.890602 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.891422 23394 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:15:45.899364 23469 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:45.849309 29366 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.906431 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.907462 23396 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:45.909914 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.913893 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.914772 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.915239 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.915598 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:45.892410 29412 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:45.892813 29575 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:45.882565 29599 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:45.956367 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:45.956972 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 304394us
I20260709 12:15:45.892385 29607 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:45.958881 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:45.897701 29591 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:45.973171 24010 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:45.973289 23469 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:45.977119 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:45.849063 29572 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:45.978549 29591 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.977209 23360 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:45.895857 29479 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:15:45.897290 29435 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 12:15:45.958766 29334 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:45.902055 29462 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 12:15:45.960961 29412 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:45.986585 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:45.989852 23469 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 11.
I20260709 12:15:45.971143 29598 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:45.988137 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.962880 29571 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:45.971422 29464 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:45.917279 29433 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:45.994791 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 169560us
I20260709 12:15:45.976953 29442 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:45.990329 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:45.994094 23393 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:45.981340 29591 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:45.959861 29366 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.992074 29412 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:45.983340 29470 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:46.041837 29470 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.977339 29437 consensus_queue.cc:1048] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:15:46.029932 28970 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:45.986049 29479 consensus_queue.cc:1243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:46.043720 28970 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:45.988600 29334 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.044660 29470 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.037900 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.976313 29581 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:15:46.045462 29442 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.044876 29334 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:46.051235 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:45.995172 29546 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:46.037786 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.049917 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.054385 23469 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:46.054603 23470 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 11. Current term is 12. Ops: []
I20260709 12:15:45.999529 29441 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:45.995433 29598 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.052188 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:46.030515 29464 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:46.079630 29595 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.079947 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.080291 29595 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.080842 23393 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:46.081799 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.082545 23393 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:46.082639 29595 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.083266 29553 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:46.083321 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:46.084043 23393 raft_consensus.cc:606] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:46.087536 23392 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:46.089017 29612 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.089609 29559 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:46.090029 29612 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.059101 29598 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:46.092639 29612 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.055997 28875 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:46.093986 28875 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.094503 28875 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.099014 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:46.123371 23390 raft_consensus.cc:2410] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:15:46.124382 23312 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 10: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:46.056783 29479 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:46.081565 29441 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:46.152989 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 117425us
I20260709 12:15:46.166286 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.065833 29313 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:15:46.178445 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.188586 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.191758 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:46.192374 23321 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:46.189519 23312 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:46.201382 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.202272 23312 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:46.207940 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.177083 29470 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 12:15:46.243855 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73134.3us
I20260709 12:15:46.202913 29553 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:46.203186 29595 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:46.239405 29546 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:46.256028 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.211089 29464 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.325457 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.339334 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.272779 29479 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:46.325191 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 75758.6us
I20260709 12:15:46.325464 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.252637 29586 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:46.351814 29592 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:46.352811 29615 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 12:15:46.353623 29630 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:15:46.353892 29583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.006s	sys 0.016s
I20260709 12:15:46.355238 29441 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.340076 29596 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:46.343675 29561 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.353929 29442 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.255712 29595 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.273104 29462 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:46.357955 29623 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:15:46.358726 29614 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:15:46.345506 29572 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:46.373980 29583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.007s	sys 0.016s
I20260709 12:15:46.374603 29552 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:46.375173 29599 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:46.377341 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:46.419144 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:46.419767 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.373183 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:46.372783 29591 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.254302 29553 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.385931 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:46.387516 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 285804us
I20260709 12:15:46.424544 23468 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:46.425383 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:46.388820 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.349990 29479 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.434162 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:46.386380 29592 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.398778 29561 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.264086 29412 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.433745 29589 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:15:46.457964 29412 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:46.380815 29482 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.005s
I20260709 12:15:46.459419 29482 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.413750 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.461529 29482 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.324381 29546 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.273536 29602 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.281s
I20260709 12:15:46.425691 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.341512 29433 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.426715 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.410188 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1074) took 1451 ms. Trace:
I20260709 12:15:46.435801 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:46.461335 23461 raft_consensus.cc:2410] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:15:46.413829 29462 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:46.467316 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 64951.9us
I20260709 12:15:46.428727 29643 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.423264 29552 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.424393 29599 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.472654 29589 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:46.471242 23395 rpcz_store.cc:276] 0709 12:15:44.936835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.946681 (+  9846us) service_pool.cc:224] Handling call
0709 12:15:44.947005 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:44.947258 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:15:44.947634 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 12:15:45.182006 (+234372us) log.cc:844] Serialized 14 byte log entry
0709 12:15:45.182356 (+   350us) log.cc:844] Serialized 14 byte log entry
0709 12:15:45.735867 (+553511us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b18003d8ca0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:46.250219 (+514352us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18003d8ca0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:46.366424 (+116205us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18003d8ca0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:46.387462 (+ 21038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:46.387732 (+   270us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:15:46.387996 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.388278 (+   282us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:46.388603 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44931,"prepare.queue_time_us":592564,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":9191,"raft.queue_time_us":82830,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":38488,"spinlock_wait_cycles":102378752,"thread_start_us":332199,"threads_started":2,"wal-append.queue_time_us":292273}
I20260709 12:15:46.464519 29642 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:46.510270 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:46.515030 29639 log.cc:927] Time spent T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:15:46.515461 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 79747.3us
I20260709 12:15:46.516901 29433 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.517490 29433 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.480778 29644 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:15:46.520187 29433 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.520763 29433 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:46.521091 29433 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.525878 29642 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:46.526818 29592 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.526937 29642 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.531334 29599 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.532862 28875 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.540010 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.540814 23321 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 10.
I20260709 12:15:46.544705 23320 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:46.546228 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.545987 29589 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.546860 23314 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:46.549883 29561 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.550802 29546 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.545980 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.552222 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.552824 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:46.553444 29552 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.555127 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:46.560832 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.561512 23314 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:46.541234 29433 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:46.597905 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:46.593067 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.595465 23320 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:46.603166 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:46.591775 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 75307.1us
I20260709 12:15:46.602011 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.590267 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:46.604858 23321 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:46.539423 29288 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:15:46.608462 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.491220 29643 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:46.596819 29334 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.000s
I20260709 12:15:46.611814 29572 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:46.608827 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:46.612341 29575 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.587841 29610 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:15:46.598743 29567 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.612265 28875 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.612845 29581 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:46.630622 29581 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:46.615928 29334 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.631035 29581 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.632560 29334 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.616298 29572 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.559010 29598 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.633570 29572 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:46.635298 29598 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.635469 29643 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.555014 29599 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
W20260709 12:15:46.616853 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 80799.8us
I20260709 12:15:46.616901 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:46.640169 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:46.644902 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.646662 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:46.624125 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:46.642601 29643 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:46.599617 29607 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:15:46.648214 29643 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:46.619275 29609 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.490s	user 0.004s	sys 0.000s
I20260709 12:15:46.648656 29643 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:46.621728 29610 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.649289 29610 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:46.655316 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:46.656917 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.616731 29561 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:46.612679 29591 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.622918 29553 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.629287 29595 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.624100 29462 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:46.624845 29479 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:46.662071 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.663096 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.667783 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.660848 29488 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:46.622067 29567 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.661491 29561 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.654160 28875 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:15:46.627729 29575 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:46.663964 29559 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:46.672374 29570 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:46.654052 29546 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:15:46.673377 29488 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:46.692412 29561 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:46.698014 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 57931.6us
I20260709 12:15:46.694602 29594 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:46.700356 29647 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:46.711361 29594 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.741915 29594 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:46.745163 29595 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:46.745600 29595 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:46.746196 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:46.746443 29595 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.746658 29559 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:46.747638 29595 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:46.748281 29479 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:46.748539 29559 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:46.758049 29585 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:46.711550 29651 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 12:15:46.772639 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2717) took 1182 ms. Trace:
W20260709 12:15:46.771010 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43665.4us
I20260709 12:15:46.773025 23463 rpcz_store.cc:276] 0709 12:15:45.590111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.592227 (+  2116us) service_pool.cc:224] Handling call
0709 12:15:45.592550 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:45.670163 (+ 77613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.771486 (+1101323us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b540044b954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:46.772571 (+  1085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176830336}
I20260709 12:15:46.775381 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:46.730459 29647 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:46.724468 29546 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.769295 29366 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.788574 29647 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:46.795724 29656 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:15:46.797816 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:46.800649 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:46.800824 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:46.785117 29607 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.801744 24014 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:46.810231 29288 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:46.843173 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:46.844966 24012 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:46.844816 29288 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.847177 29288 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:46.812964 29334 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.848459 29334 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.849668 29366 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:46.841941 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2887) took 1852 ms (client timeout 1895 ms). Trace:
I20260709 12:15:46.850778 29334 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.851578 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:46.850811 23390 rpcz_store.cc:269] 0709 12:15:44.959165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:44.960506 (+  1341us) service_pool.cc:224] Handling call
0709 12:15:46.811463 (+1850957us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002dfd54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:46.811613 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2565347712}
W20260709 12:15:46.854327 29660 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:46.801947 29412 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.857213 29643 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.802908 29651 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:46.797567 29464 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:46.792768 29546 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.813876 29625 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:15:46.860633 24012 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
W20260709 12:15:46.861951 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 51389.4us
I20260709 12:15:46.769757 29488 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:46.834501 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.813247 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:46.869750 24008 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:46.869889 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:46.872514 23461 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 19: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:46.811731 29642 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:46.881188 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:46.858176 29412 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.885746 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.886904 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.887409 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:46.888087 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.881716 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.896296 23463 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:15:46.900110 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.901893 23316 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:46.902482 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:46.903167 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:46.844018 23215 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 319) took 18295 ms (client timeout 15000 ms). Trace:
I20260709 12:15:46.898804 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.906262 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.913345 23390 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:46.910614 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.913960 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:46.914346 23395 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:46.801337 29610 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.908582 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:46.923112 23388 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:15:46.923548 23388 pending_rounds.cc:85] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 8
I20260709 12:15:46.926236 23388 pending_rounds.cc:107] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:15:46.926600 23388 raft_consensus.cc:2889] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:46.861255 29561 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.928690 24011 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:46.861297 29607 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:46.933177 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 142734us
I20260709 12:15:46.935053 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2720) took 1285 ms. Trace:
I20260709 12:15:46.935398 23470 rpcz_store.cc:276] 0709 12:15:45.650278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.650590 (+   312us) service_pool.cc:224] Handling call
0709 12:15:45.655032 (+  4442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.058965 (+403933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.934890 (+875925us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:46.935010 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910643584}
I20260709 12:15:46.909103 29651 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:46.861452 29464 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:46.937768 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:46.944161 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.940730 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.920217 29599 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:46.897491 29642 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:46.922020 29488 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:46.920415 29559 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:46.971278 29572 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:46.973415 29572 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.973809 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.975427 23467 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:15:46.976445 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:46.977888 28970 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.982905 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:46.986838 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:46.994385 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:46.996541 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.003400 24012 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:47.011164 24011 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:46.947649 29651 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:47.030786 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.033583 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:46.969935 29642 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:46.950095 29567 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:46.966460 29599 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:47.038196 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 106538us
I20260709 12:15:47.030447 29504 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:47.054632 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.061532 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.065624 29599 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.065764 29504 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.068989 29504 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.041009 29651 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:47.075513 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.076215 23389 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:47.078131 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2719) took 1468 ms. Trace:
I20260709 12:15:47.078398 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1651) took 1276 ms. Trace:
I20260709 12:15:47.078539 23462 rpcz_store.cc:276] 0709 12:15:45.610143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.611456 (+  1313us) service_pool.cc:224] Handling call
0709 12:15:45.611906 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:45.796655 (+184749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.077887 (+1281232us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54002a08d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.078070 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423498496}
I20260709 12:15:47.078773 23318 rpcz_store.cc:276] 0709 12:15:45.801606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.801855 (+   249us) service_pool.cc:224] Handling call
0709 12:15:45.807608 (+  5753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.051020 (+243412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:46.051065 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:46.051128 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:15:46.051184 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.078170 (+1026986us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400331b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.078216 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.078320 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559187200}
I20260709 12:15:47.080447 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:47.082181 29596 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:47.082813 29596 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.086910 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.095327 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:47.096014 29604 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:47.098124 29667 log.cc:927] Time spent T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:15:47.097872 29604 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:47.100035 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 67179.9us
I20260709 12:15:47.100735 29604 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.102541 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.103055 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.113759 29464 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.117633 23462 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:47.118935 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.119756 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:47.120827 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.115633 29599 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.122581 29610 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.124478 29599 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.124878 29412 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.014036 29559 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:47.104156 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:47.138115 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 218873us
I20260709 12:15:47.019814 29553 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:47.140092 23467 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:15:47.146459 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.146299 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:47.147521 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.148774 24014 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:47.110585 29561 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.151800 29553 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.152287 24010 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:47.154066 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.153084 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.138742 29479 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:47.021833 29581 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.163353 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:47.105113 29546 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:47.153525 29666 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:15:47.142030 29504 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.142261 29144 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:47.189616 29676 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:47.189332 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.190578 24008 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:47.191439 29676 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:47.192691 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.194144 29672 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:47.195675 29610 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:47.196308 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:47.198673 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.199419 29676 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:47.199685 29610 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:47.199959 29676 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:47.200587 29676 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:47.209646 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:47.112098 29596 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.106169 29651 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.165091 29561 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:47.230206 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 97277.9us
I20260709 12:15:47.229926 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.174384 29572 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.234596 23469 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:47.182454 29546 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.235819 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:47.239540 24010 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:47.232226 29144 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:46.912572 23215 rpcz_store.cc:269] 0709 12:15:28.548170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:28.548378 (+   208us) service_pool.cc:224] Handling call
0709 12:15:46.843733 (+18295355us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:42.584221 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:42.584766 (+   545us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:42.584815 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:15:42.584835 (+    20us) tablet.cc:662] Decoding operations
0709 12:15:42.586249 (+  1414us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:42.586332 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:42.586364 (+    32us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:15:42.696720 (+110356us) tablet.cc:701] Row locks acquired
0709 12:15:42.698888 (+  2168us) write_op.cc:260] PREPARE: finished
0709 12:15:42.717347 (+ 18459us) write_op.cc:270] Start()
0709 12:15:42.724071 (+  6724us) write_op.cc:276] Timestamp: P: 1783599342714392 usec, L: 0
0709 12:15:42.728974 (+  4903us) op_driver.cc:348] REPLICATION: starting
0709 12:15:42.845642 (+116668us) log.cc:844] Serialized 5979 byte log entry
0709 12:15:43.439883 (+594241us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:43.496993 (+ 57110us) op_driver.cc:464] REPLICATION: finished
0709 12:15:43.498273 (+  1280us) write_op.cc:301] APPLY: starting
0709 12:15:43.498366 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:43.842867 (+344501us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:43.842923 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:43.939943 (+ 97020us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:43.940389 (+   446us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 12:15:43.940460 (+    71us) write_op.cc:312] APPLY: finished
0709 12:15:43.941174 (+   714us) log.cc:844] Serialized 139 byte log entry
0709 12:15:43.952273 (+ 11099us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:43.952595 (+   322us) write_op.cc:454] Released schema lock
0709 12:15:43.952700 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:43.969261 (+ 16561us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:43.969291 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":179670,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":992,"apply.run_cpu_time_us":26779,"apply.run_wall_time_us":470882,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66887,"num_ops":15,"prepare.queue_time_us":344740,"prepare.run_cpu_time_us":9091,"prepare.run_wall_time_us":403624,"raft.queue_time_us":28226,"raft.run_cpu_time_us":1126,"raft.run_wall_time_us":1198,"replication_time_us":767856,"spinlock_wait_cycles":150045952,"thread_start_us":401764,"threads_started":4,"wal-append.queue_time_us":191463,"wal-append.run_cpu_time_us":14497,"wal-append.run_wall_time_us":1927805}]]}
I20260709 12:15:47.248027 29144 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.248736 23390 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:47.186808 29647 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.212134 29504 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.255460 29504 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.256103 29546 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.258477 29546 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.260953 29479 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:15:47.262207 29669 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:15:47.267251 29462 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:47.268158 29652 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:47.270049 29647 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:47.270902 29650 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.664s	user 0.001s	sys 0.002s
I20260709 12:15:47.271457 29652 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:47.271908 29652 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:47.273767 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1086) took 1281 ms. Trace:
I20260709 12:15:47.274258 23394 rpcz_store.cc:276] 0709 12:15:45.992712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.096301 (+103589us) service_pool.cc:224] Handling call
0709 12:15:46.096721 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:46.097171 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:46.097203 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:46.138837 (+ 41634us) log.cc:844] Serialized 52 byte log entry
0709 12:15:46.435602 (+296765us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:46.484376 (+ 48774us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.508492 (+ 24116us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.555857 (+ 47365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:46.555956 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:47.270195 (+714239us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:47.273506 (+  3311us) raft_consensus.cc:1685] finished
0709 12:15:47.273539 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.273698 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54195,"prepare.queue_time_us":109160,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":5250,"raft.queue_time_us":55528,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":45845,"spinlock_wait_cycles":48473472,"thread_start_us":264337,"threads_started":1,"wal-append.queue_time_us":381594}
I20260709 12:15:47.275892 29652 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.279299 29559 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.289319 29677 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:47.289512 29652 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:47.289889 29677 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:47.290341 29677 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:47.282022 29595 raft_consensus.cc:993] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:15:47.292106 29595 raft_consensus.cc:1081] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:15:47.284164 29581 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:47.294452 29433 consensus_meta.cc:287] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.002s
I20260709 12:15:47.294939 29433 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.296814 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.277153 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.298743 23394 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:47.294296 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.299758 23390 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:47.236310 29561 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.317330 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.319056 29561 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.297178 29433 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.236076 29442 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:47.320428 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110841us
I20260709 12:15:47.331363 23394 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:47.331648 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:47.302042 29596 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:47.303673 29594 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.561s	user 0.003s	sys 0.008s
I20260709 12:15:47.336174 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.332376 23390 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:15:47.338189 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.339597 23390 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:47.238445 29572 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.310688 29366 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:47.353875 29366 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.357015 29366 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.346675 29553 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.346365 29676 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.338371 29607 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.348829 29651 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.375555 29651 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.376060 29651 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.378283 29651 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.336319 29442 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.380218 29442 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.339322 29594 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.350183 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.382299 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.382814 29594 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.382887 23392 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:47.383913 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.384735 23392 raft_consensus.cc:2368] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:47.345518 29479 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.371623 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:47.400434 23470 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.373860 29676 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.337261 29561 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:47.336892 23394 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:47.342782 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.411972 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.407048 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.412876 23392 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:47.413779 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.418891 23390 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:47.427819 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.429371 23392 raft_consensus.cc:2376] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14. Re-sending same reply.
I20260709 12:15:47.449627 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.450881 23321 pending_rounds.cc:85] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 8
I20260709 12:15:47.451231 23321 pending_rounds.cc:107] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:15:47.449800 29366 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:47.451462 23321 raft_consensus.cc:2889] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:47.451751 29366 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.453899 29366 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.451474 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:47.455441 23317 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:47.460230 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.462460 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1305) took 1074 ms. Trace:
I20260709 12:15:47.463090 23466 rpcz_store.cc:276] 0709 12:15:46.387978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.388087 (+   109us) service_pool.cc:224] Handling call
0709 12:15:46.413800 (+ 25713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.532092 (+118292us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:46.676400 (+144308us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.771809 (+ 95409us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:46.771858 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:46.771909 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.462030 (+690121us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000c7454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.462134 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.462375 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":175575,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":930,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11076,"spinlock_wait_cycles":271501312,"thread_start_us":238926,"threads_started":2,"wal-append.queue_time_us":206866}
I20260709 12:15:47.409564 29442 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:47.467509 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.467144 29442 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.469025 23466 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:47.418313 29561 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:47.466423 29670 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.410s	user 0.005s	sys 0.000s
I20260709 12:15:47.475394 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.476399 23392 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:47.476999 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.482288 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45800 (request call id 53) took 1441 ms. Trace:
I20260709 12:15:47.484941 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.487280 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.482757 23393 rpcz_store.cc:276] 0709 12:15:46.030245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.083185 (+ 52940us) service_pool.cc:224] Handling call
0709 12:15:47.471145 (+1387960us) spinlock_profiling.cc:243] Waited 989 ms on lock 0x7b54001142a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95f1520 00007ff0a966e652 00007ff0ae63c60a 00007ff0a7bc27d4 00007ff0a7bc2762 00007ff0a7bc2691 00007ff0a7bc260c 00007ff0a7bc18c2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.471386 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2274157312}
I20260709 12:15:47.496740 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.418499 29479 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.499241 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.500226 23393 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:47.500734 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.482064 29561 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:47.508638 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:47.510064 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.510787 23394 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:47.507699 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.512121 23393 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:47.513100 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.513777 23393 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:15:47.515004 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.515802 23393 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:15:47.493979 29591 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:47.522349 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:47.525161 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.525488 23318 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.523465 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.529359 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:47.530354 29442 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.530791 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:47.530959 29442 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:47.531746 29442 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:47.532115 29442 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:47.532473 29442 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:47.534389 29677 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.536623 29677 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.537918 29673 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:15:47.542730 29479 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.546720 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.548513 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.549131 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1090) took 1493 ms. Trace:
I20260709 12:15:47.549707 23462 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:47.551210 23387 rpcz_store.cc:276] 0709 12:15:46.055779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.546289 (+490510us) service_pool.cc:224] Handling call
0709 12:15:46.546598 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:46.638303 (+ 91705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:46.638335 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:46.645946 (+  7611us) log.cc:844] Serialized 26 byte log entry
0709 12:15:46.889473 (+243527us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:46.889531 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.548739 (+659208us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b54002aeed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.548788 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:47.548831 (+    43us) raft_consensus.cc:1685] finished
0709 12:15:47.548846 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.549074 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5543,"raft.queue_time_us":2586,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":6581,"spinlock_wait_cycles":209476864,"thread_start_us":223742,"threads_started":1,"wal-append.queue_time_us":329218}
I20260709 12:15:47.550726 23466 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:47.558477 23393 raft_consensus.cc:2393] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:47.559549 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:47.550040 23395 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:47.560956 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.564497 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.566188 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.566728 23387 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:47.567507 23395 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:15:47.569070 23395 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:47.570233 23396 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:15:47.570396 23395 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:47.567489 23394 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:47.574813 23394 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:47.522516 29642 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.586418 23470 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:47.529289 29591 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.588176 23395 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:15:47.589499 23394 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:47.589926 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.591080 23396 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:47.558537 29652 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:47.565546 29644 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.627202 23395 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:15:47.628286 23318 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:47.628890 23468 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:47.580286 29674 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:47.632712 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1310) took 1205 ms. Trace:
I20260709 12:15:47.731333 23387 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:47.638559 23465 rpcz_store.cc:276] 0709 12:15:46.427375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.427493 (+   118us) service_pool.cc:224] Handling call
0709 12:15:46.427836 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:46.513083 (+ 85247us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:46.675926 (+162843us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.783924 (+107998us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:46.783989 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:46.784044 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.632433 (+848389us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b54001883d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:47.632492 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.632643 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10332,"prepare.queue_time_us":185722,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":48,"raft.queue_time_us":11530,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3918,"spinlock_wait_cycles":195547008,"thread_start_us":247490,"threads_started":2,"wal-append.queue_time_us":177589}
W20260709 12:15:47.742751 23360 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.653s (SENT) [suppressed 15 similar messages]
I20260709 12:15:47.629102 29642 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:47.735491 29656 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:15:47.767427 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1654) took 1965 ms. Trace:
I20260709 12:15:47.767758 29646 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:47.767947 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 398482us
I20260709 12:15:47.727337 29572 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:47.779307 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.770254 23319 rpcz_store.cc:276] 0709 12:15:45.802462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:45.802872 (+   410us) service_pool.cc:224] Handling call
0709 12:15:45.803179 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:45.803395 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:15:45.976410 (+173015us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.130980 (+154570us) log.cc:844] Serialized 14 byte log entry
0709 12:15:46.753461 (+622481us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b180038e780. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:15:47.547100 (+793639us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b180038e780. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:15:47.745950 (+198850us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:47.752288 (+  6338us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:47.752342 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:47.752406 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:47.767373 (+ 14967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9728,"prepare.queue_time_us":239200,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":106506,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":12477,"spinlock_wait_cycles":11595264,"thread_start_us":519641,"threads_started":3,"wal-append.queue_time_us":218911,"wal-append.run_cpu_time_us":1368,"wal-append.run_wall_time_us":1010019}
I20260709 12:15:47.794814 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:47.795439 29202 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:47.797748 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.798558 24010 raft_consensus.cc:2368] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13.
I20260709 12:15:47.800427 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.797032 29202 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:47.801630 29202 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:47.801985 29687 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:15:47.802749 29559 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:47.802618 29202 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.803310 29559 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.804446 29598 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.804948 29598 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:47.805542 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:47.805824 24014 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:15:47.806671 29596 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.807008 29598 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.807121 29671 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:47.807655 29671 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.808523 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:47.810676 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.801838 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.812564 24015 raft_consensus.cc:2393] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:47.726636 29644 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.811712 24012 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:47.822330 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.783739 29570 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.831025 23396 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:47.828788 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.833025 23313 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:47.829856 24014 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:47.727553 29652 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.835561 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.831899 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.844295 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.846385 23318 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:47.805363 29559 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:47.848326 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.818225 29488 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.853108 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.779250 29646 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.853978 24015 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:47.855437 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:47.858462 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:47.836935 29594 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:47.863674 23466 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:47.864703 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:47.865669 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.867328 24008 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:47.807114 29596 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.869860 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.875237 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.875454 23392 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:47.879021 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.850492 29677 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:47.887754 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.891148 23392 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:47.883561 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.892179 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:47.892391 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:47.897807 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.901310 23392 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:47.901806 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.902318 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:47.903056 23392 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:47.843367 29570 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.869673 29591 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.914865 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.883054 29610 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:47.917634 24015 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:47.859903 29488 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:47.894016 29671 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:47.896117 29607 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:15:47.927155 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 184417us
I20260709 12:15:47.869556 29594 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.927421 24007 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:47.929647 24012 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:47.938550 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.922631 29572 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:47.926316 23980 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:47.921039 29688 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:15:47.939574 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.905489 29561 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:47.944582 23396 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:47.942772 24015 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:15:47.904398 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:47.947072 23392 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:47.948598 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.895553 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:47.948882 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:47.951371 23392 raft_consensus.cc:606] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.931236 23360 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.952240 23392 raft_consensus.cc:614] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:47.964145 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:47.936623 29607 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:47.973006 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.993328 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.994925 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:47.991489 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:47.964541 29437 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:47.967931 29610 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:47.967209 29580 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:47.990537 29671 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:48.017712 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:47.998689 29589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.387s	user 0.020s	sys 0.043s
I20260709 12:15:48.021386 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.023502 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.024251 23463 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:48.027560 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.025259 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:48.020792 29589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.020s	sys 0.044s
I20260709 12:15:48.029886 23463 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:48.009912 29644 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.029759 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.014276 29482 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:15:48.023036 29690 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:15:48.039362 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.037856 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.035035 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.040266 23462 raft_consensus.cc:2410] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:15:48.040475 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:48.045871 23312 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:48.046710 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:48.047546 23392 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:48.048695 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:48.049770 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.050577 23463 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:15:48.052400 23436 leader_election.cc:400] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13.
I20260709 12:15:48.052826 23436 leader_election.cc:403] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:48.057143 29677 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:48.058246 29677 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.061738 23392 raft_consensus.cc:2368] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:48.064530 23392 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:48.068783 23462 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:15:48.031029 29671 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:48.072223 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 180153us
I20260709 12:15:48.072762 23462 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:48.072922 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.014266 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:48.078025 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63861.9us
I20260709 12:15:48.079003 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.084682 23463 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:48.085896 23392 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:48.043660 29652 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.045468 29482 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:48.094414 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.070323 29646 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.099697 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.090158 29581 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:48.087244 29479 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13.
W20260709 12:15:48.105849 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 53661.4us
I20260709 12:15:48.106941 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.090170 23467 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:15:48.107800 23467 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:15:48.107956 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.108197 23467 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:48.115489 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.115751 24007 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:48.103339 29202 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:48.102933 29570 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:48.102444 29488 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.051937 29607 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.122892 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.109850 29581 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:48.111593 29437 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:48.123529 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.126256 24012 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:48.126920 29437 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.127307 29437 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:48.127417 29643 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.116493 29580 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:48.116672 29462 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:48.134944 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.133457 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.136920 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.111485 29412 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:15:48.136965 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.140661 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.134670 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.137374 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.141652 23312 raft_consensus.cc:2368] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15.
I20260709 12:15:48.138828 23319 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:48.142441 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.142807 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.113185 29652 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:15:48.144542 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.122972 29697 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:48.141273 23313 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:48.145891 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.146167 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.147701 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.174321 23317 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:15:48.153714 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2905) took 1179 ms. Trace:
I20260709 12:15:48.153959 29695 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:48.154991 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.155918 29437 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.163897 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.139137 29482 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.142967 29680 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:48.174804 23393 rpcz_store.cc:276] 0709 12:15:46.974256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.562369 (+588113us) service_pool.cc:224] Handling call
0709 12:15:47.562691 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:47.564070 (+  1379us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:47.858960 (+294890us) log.cc:844] Serialized 14 byte log entry
0709 12:15:48.072304 (+213344us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:48.087825 (+ 15521us) log.cc:844] Serialized 26 byte log entry
0709 12:15:48.087997 (+   172us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:48.088045 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:48.088147 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:48.153508 (+ 65361us) raft_consensus.cc:1685] finished
0709 12:15:48.153529 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:48.153657 (+   128us) inbound_call.cc:177] Queueing success response
0709 12:15:48.165783 (+ 12126us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b4800002dd8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
Metrics: {"prepare.queue_time_us":331810,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":6150,"raft.queue_time_us":183,"raft.run_cpu_time_us":2501,"raft.run_wall_time_us":105799,"spinlock_wait_cycles":7577984,"thread_start_us":408209,"threads_started":2,"wal-append.queue_time_us":241310}
I20260709 12:15:48.178658 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.171078 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.171366 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.179280 23313 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:48.137485 29561 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:48.119820 29671 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.134791 29594 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:48.184718 29674 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:48.173678 23360 leader_election.cc:400] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15.
I20260709 12:15:48.132474 29581 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:48.133999 29479 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:48.186029 29646 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:48.139858 29580 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:48.185809 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:48.185230 29674 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.185266 23467 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.188665 23360 leader_election.cc:403] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:15:48.172072 29442 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.005s
I20260709 12:15:48.195441 23463 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:48.186506 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.192484 23317 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 12.
I20260709 12:15:48.197774 29646 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:48.194659 29462 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:48.199791 29318 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:48.200191 29313 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:48.197568 29442 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:48.201892 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:48.202629 29646 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:48.203100 29646 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:48.197111 29479 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:15:48.205323 29479 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:48.205683 29479 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:48.206948 29437 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:48.206558 29479 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.204129 29580 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.207806 29479 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:48.212534 23467 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:48.220238 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.203987 29646 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.221015 24012 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:48.221897 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.222702 24015 raft_consensus.cc:2410] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:15:48.226845 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:48.234998 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.235707 24015 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:48.235476 29589 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:48.234853 29318 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:48.236761 29318 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:48.237023 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:48.237728 29689 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.392s	user 0.002s	sys 0.000s
I20260709 12:15:48.238152 29610 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:48.238344 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:48.238672 29610 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:48.237130 29318 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:48.239346 29366 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:48.239594 29610 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:48.237752 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 164897us
I20260709 12:15:48.241952 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.240046 29366 raft_consensus.cc:2764] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:15:48.242789 24012 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:48.243995 24014 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:48.246234 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.247942 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:48.248504 24016 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:48.248255 29652 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:48.249547 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.227633 29462 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:48.255391 24008 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:48.255412 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.263276 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.204170 29442 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:48.278402 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 104877us
I20260709 12:15:48.280566 23388 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:48.281739 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:48.282328 23389 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:48.282609 23389 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:48.283360 23389 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.284546 23389 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:48.286067 23391 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:48.228824 29433 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:15:48.286242 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:48.281785 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.287240 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:48.304634 23395 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:48.306939 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:48.228940 29651 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:15:48.311221 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:48.313616 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.313877 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:48.315434 24016 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:48.266523 29697 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:48.264819 29462 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.332047 23387 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:48.318244 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56031.2us
I20260709 12:15:48.224371 29581 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.282619 29680 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:48.334924 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.333271 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:48.337090 23360 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:48.341056 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.240070 29318 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.344329 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.345016 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:48.347620 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.311319 29607 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15.
I20260709 12:15:48.350234 23461 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:48.349393 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:48.351707 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:48.353905 23387 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:48.323498 29479 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:48.320009 29677 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:48.354508 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.357112 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:48.357822 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.359298 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.337033 29462 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:48.389958 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:48.392000 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:48.388687 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53883.4us
I20260709 12:15:48.395381 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:48.435483 23391 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:48.342736 29643 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:48.349875 29559 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:15:48.437788 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 45883.8us
W20260709 12:15:48.442342 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 128563us
W20260709 12:15:48.437125 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 82678.8us
I20260709 12:15:48.445314 23362 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.446832 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.356057 29433 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:48.443714 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:48.356842 29594 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:48.361682 29671 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:48.392165 29707 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:48.435118 29644 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:48.475889 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 134956us
I20260709 12:15:48.478332 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.452935 29600 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:48.460949 29607 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:48.440637 29318 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.461462 29433 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:48.442358 29559 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:48.480611 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.457584 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.441522 29704 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:48.483350 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 88051.5us
I20260709 12:15:48.484047 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.506523 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:48.395750 29561 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:48.470706 29594 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.469571 29643 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:48.499830 23467 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:48.497520 23389 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:48.504689 23387 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:15:48.466856 29479 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:15:48.458811 29680 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:15:48.516613 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.482138 29572 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:15:48.481925 29600 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:48.480849 29671 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:48.521102 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:48.521386 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 63905.5us
I20260709 12:15:48.488924 29580 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:48.533387 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.506898 29559 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:48.547755 29559 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:48.551030 29646 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:48.510239 29644 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:48.512812 29594 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:48.563577 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.564524 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.565143 24016 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:15:48.562976 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.572118 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:48.505630 29589 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:15:48.518149 29482 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:48.480505 29674 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:48.561396 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.526038 29651 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:15:48.577165 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.531454 29600 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:48.550452 29707 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:48.561241 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 109325us
I20260709 12:15:48.575477 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:48.562763 29644 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:48.590410 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.681013 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.571250 29680 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:48.570513 29572 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:48.568241 29488 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:48.591322 29581 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:48.609113 29561 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:48.612176 29644 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:15:48.714465 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 137392us
I20260709 12:15:48.682447 29646 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:48.715763 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.682998 29677 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:48.691496 29707 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:48.722985 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:48.685081 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 101804us
W20260709 12:15:48.697213 29718 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:15:48.578843 29433 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:48.736554 29714 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
W20260709 12:15:48.737831 29719 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 12:15:48.708083 29651 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:15:48.737546 29737 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:48.854089 29730 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:48.789780 29433 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:48.862795 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 147110us
I20260709 12:15:48.717242 29488 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:15:48.710372 29671 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.877280 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.871737 23461 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:48.878937 23467 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:48.879889 23979 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.150.254:44861, user_credentials={real_user=slave}} blocked reactor thread for 62944.6us
I20260709 12:15:48.862205 29559 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:48.882130 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.888479 23287 leader_election.cc:400] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 11. Current term is 12.
I20260709 12:15:48.889642 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.891919 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.870550 29746 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:15:48.893824 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 203914us
I20260709 12:15:48.896922 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.857009 29740 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:15:48.861985 29572 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:48.866007 29646 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:48.884032 29610 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:48.887149 29724 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:48.894027 29674 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:48.895310 29747 raft_consensus.cc:2764] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:15:48.946561 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49720us
W20260709 12:15:48.913244 29728 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:15:48.952636 23320 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:48.952689 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.896590 29642 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:48.977994 23388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.016s	sys 0.059s
I20260709 12:15:48.920717 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:48.881102 29730 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:48.896250 29746 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:48.980154 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59527.8us
W20260709 12:15:48.915589 29729 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 12:15:48.917872 29726 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 12:15:48.979488 23388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.017s	sys 0.060s
W20260709 12:15:48.951588 29599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.540s	user 0.008s	sys 0.003s
I20260709 12:15:48.945422 29589 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:48.985497 23317 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
W20260709 12:15:48.986841 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:48.919401 29724 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:48.917303 29610 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:48.917810 29748 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:48.985497 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:48.990847 29713 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.629s	user 0.006s	sys 0.015s
I20260709 12:15:48.990727 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:48.988516 29746 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:48.988180 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:48.920888 29737 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 12:15:48.991360 29599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.625s	user 0.008s	sys 0.004s
I20260709 12:15:48.982240 29567 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:49.006175 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 283269us
I20260709 12:15:49.006620 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:48.977730 29731 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 12:15:49.017069 29745 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:15:48.998458 29677 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:48.996999 29724 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:49.008074 29704 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.031152 29749 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:49.031643 24015 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:15:48.980677 29752 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:15:48.996590 29707 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:49.036659 29318 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.000s
I20260709 12:15:48.998416 29589 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.046931 29594 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:49.049389 29462 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.003s
I20260709 12:15:49.053668 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 1046) took 1248 ms. Trace:
I20260709 12:15:49.100369 24007 rpcz_store.cc:276] 0709 12:15:47.805310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.808269 (+  2959us) service_pool.cc:224] Handling call
0709 12:15:49.053399 (+1245130us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54005ed654. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:49.053582 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271556992}
W20260709 12:15:49.078648 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 72108.8us
I20260709 12:15:49.078794 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2752) took 1008 ms. Trace:
I20260709 12:15:49.102212 23463 rpcz_store.cc:276] 0709 12:15:48.070087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.104897 (+ 34810us) service_pool.cc:224] Handling call
0709 12:15:48.120196 (+ 15299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.195843 (+ 75647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.078546 (+882703us) spinlock_profiling.cc:243] Waited 75.1 ms on lock 0x7b540044af54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:49.078737 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172650880}
I20260709 12:15:49.102981 29695 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:15:49.090109 29717 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.650s	user 0.002s	sys 0.000s
I20260709 12:15:49.100389 29610 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.104900 29610 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:49.082675 29581 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:49.105010 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.105582 29581 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:49.106474 29581 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.109265 29561 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:49.109612 29567 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:49.080387 29572 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.110178 29567 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:49.110522 29572 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.111444 29730 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:49.111080 29567 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.112378 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2920) took 1304 ms. Trace:
I20260709 12:15:49.112859 29572 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.089993 29677 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.075394 29644 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.103389 29695 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.112746 23396 rpcz_store.cc:276] 0709 12:15:47.808355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.895817 (+ 87462us) service_pool.cc:224] Handling call
0709 12:15:49.112205 (+1216388us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400058554. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:49.112318 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66511616}
I20260709 12:15:49.115974 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:15:49.123925 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.125152 23390 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:49.127240 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:49.117177 29462 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.128913 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.130360 29600 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.130666 29677 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.130671 24007 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:49.132366 23470 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 22: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:49.139009 23318 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:49.146322 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.132112 29462 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.135469 29598 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:49.098523 29707 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:49.161528 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 84619.8us
I20260709 12:15:49.161684 29756 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:49.162281 29707 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.162340 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:49.109637 29589 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.163046 29652 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:49.163441 29652 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.163519 29755 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.164011 23436 leader_election.cc:400] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:49.164479 23436 leader_election.cc:403] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:49.164062 29755 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.164316 29652 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.167114 29755 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.094358 29704 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.168895 29697 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:49.134222 29764 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:49.154493 29594 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:49.155333 29736 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:15:49.155808 29479 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:49.156416 29740 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.186197 29644 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.156659 29655 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:49.188163 29655 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.185335 29740 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.189571 29655 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.190445 29740 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:49.159575 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:49.160233 29598 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:49.162004 23316 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:15:49.193601 23469 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:49.152549 29695 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.151494 29724 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.170961 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.172284 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.197180 23462 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:49.173041 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:49.173234 23463 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:15:49.198119 23463 pending_rounds.cc:85] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 12
I20260709 12:15:49.198374 23463 pending_rounds.cc:107] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:15:49.198619 23463 raft_consensus.cc:2889] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:49.173734 23312 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:15:49.174202 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.169389 29697 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.191285 29655 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:49.176710 29313 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:15:49.180248 29591 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:15:49.178478 29479 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.178328 29680 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.178387 29652 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:49.162820 29756 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:49.194026 23469 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.201274 23362 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:49.202533 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.203765 24009 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:49.177157 29651 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:49.212325 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2920) took 2057 ms. Trace:
I20260709 12:15:49.213064 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.217754 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.227947 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:49.196038 29462 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:49.204384 29594 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:49.219062 29313 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:49.242066 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.195883 29764 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.207152 29695 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:49.217796 29598 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.221510 29724 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.188123 29704 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.241875 29313 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.226815 23316 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:49.238847 29570 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:49.233896 29488 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:49.210070 29479 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:49.244475 23462 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:49.224036 23320 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:49.226487 23464 rpcz_store.cc:276] 0709 12:15:47.155345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.340309 (+184964us) service_pool.cc:224] Handling call
0709 12:15:47.402380 (+ 62071us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:47.522785 (+120405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:47.522828 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:47.526964 (+  4136us) log.cc:844] Serialized 26 byte log entry
0709 12:15:48.235076 (+708112us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18003779e0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:48.337558 (+102482us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:48.337616 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.211920 (+874304us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54004056d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:49.211994 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:49.212058 (+    64us) raft_consensus.cc:1685] finished
0709 12:15:49.212078 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.212254 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2148,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":3805,"spinlock_wait_cycles":279143936,"thread_start_us":102174,"threads_started":1,"wal-append.queue_time_us":936673}
I20260709 12:15:49.207998 29740 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:15:49.257746 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:49.271677 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.273586 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:49.286442 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 85640.1us
I20260709 12:15:49.290095 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:49.290988 23316 raft_consensus.cc:3060] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:49.292845 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.275550 29748 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:49.275112 29488 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:49.286741 29600 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.292554 29594 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.325541 29313 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.326840 29313 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:49.303345 23464 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:15:49.329444 23464 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:49.296258 29704 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:49.330682 29704 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.331090 29704 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:49.248270 29697 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.334065 29697 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:49.302399 29764 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.273234 29680 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.338380 29725 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:15:49.322507 29488 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.340389 29770 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:15:49.324864 29594 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.343048 29594 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.345093 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.346545 23318 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:15:49.352794 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 114912us
I20260709 12:15:49.355384 23316 raft_consensus.cc:2468] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:49.356675 23467 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:49.357759 29594 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.361577 29779 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:49.362015 29594 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.363878 29694 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:49.364274 29594 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.350032 29740 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:15:49.299542 29756 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:15:49.364290 29694 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.367134 29694 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.367949 29591 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:15:49.368510 29694 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:49.370249 29746 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.371812 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:49.372928 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.364888 24008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1286) took 1242 ms. Trace:
I20260709 12:15:49.373888 24008 rpcz_store.cc:276] 0709 12:15:48.123695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.123831 (+   136us) service_pool.cc:224] Handling call
0709 12:15:48.124078 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.264904 (+140826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.364588 (+1099684us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400780d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:49.364794 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301560192}
I20260709 12:15:49.375197 23316 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:49.376111 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:49.376688 23318 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:49.376127 29676 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:15:49.385746 29707 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:49.385900 23362 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:49.387465 29746 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.387929 29707 raft_consensus.cc:2804] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:49.388679 23317 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:49.391444 23466 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:15:49.394064 29775 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:49.395280 29775 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:49.395605 29775 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.396103 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.388432 29707 raft_consensus.cc:697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:49.398447 29707 consensus_queue.cc:237] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.402661 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.403411 23391 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:49.405347 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.406240 23395 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:49.406883 23395 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 1105) took 1121 ms. Trace:
I20260709 12:15:49.407272 23395 rpcz_store.cc:276] 0709 12:15:48.285179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.323163 (+ 37984us) service_pool.cc:224] Handling call
0709 12:15:49.405161 (+1081998us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b58001d0798. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a654fa22 00007ff0af4688d9 00007ff0ae5c2ee0 00007ff0ae68c0ab 00007ff0ae68c131 00007ff0ae63a0a0 00007ff0ae639984 00007ff0a7bb46d4 00007ff0a7bb4662
0709 12:15:49.406808 (+  1647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3824512}
I20260709 12:15:49.412652 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.414395 23394 raft_consensus.cc:2410] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:15:49.416911 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.417178 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.417704 23387 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:49.419067 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47322.5us
I20260709 12:15:49.419309 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.406263 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.421788 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:49.422470 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.423290 23395 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:49.424603 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:49.425400 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.428331 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.428941 23394 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:49.429311 23389 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:49.425608 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.429481 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.433519 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.433957 23393 raft_consensus.cc:2393] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 22: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:49.435829 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.445237 24008 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:49.432247 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:49.449636 23393 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:49.451259 29721 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.451818 29721 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.453727 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.459810 24007 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:49.463953 23318 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:49.464258 23387 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:49.467523 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.468688 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.473294 23394 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:49.475394 23395 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:49.476035 23389 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:49.476617 23395 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:15:49.455924 29580 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:49.396461 29775 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.485855 23390 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:49.486128 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:49.458730 29441 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:49.488137 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.489089 23391 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:49.491321 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:49.495496 23318 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:49.496109 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:49.497390 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:15:49.498098 23393 raft_consensus.cc:606] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.498422 23393 raft_consensus.cc:614] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:15:49.498661 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.481902 29580 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:49.508831 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:49.489557 29441 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.513558 29782 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:49.501106 29674 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:49.501461 29599 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:49.516570 29724 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:49.549180 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.515372 29775 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:49.514369 29580 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:49.556674 23393 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:15:49.516496 29441 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:49.602520 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 84987us
W20260709 12:15:49.603515 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.614703 29692 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:49.615527 29781 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.616076 29781 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.620814 23316 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:49.626657 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.627296 23391 raft_consensus.cc:2376] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15. Re-sending same reply.
I20260709 12:15:49.557639 29591 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:15:49.634788 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.635581 23391 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 13: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:49.646276 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.605952 29777 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.258s
I20260709 12:15:49.645756 29596 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.606344 29599 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:15:49.656291 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 140515us
I20260709 12:15:49.633867 29479 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.656275 29779 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:49.669288 29779 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.676829 29781 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:49.679042 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.686220 29462 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:49.686712 29462 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.687561 29462 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.688508 29462 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:49.693382 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 1049) took 1142 ms. Trace:
I20260709 12:15:49.693897 24012 rpcz_store.cc:276] 0709 12:15:48.550864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.553636 (+  2772us) service_pool.cc:224] Handling call
0709 12:15:48.554024 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.589819 (+ 35795us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:15:48.924473 (+334654us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.079104 (+154631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:49.079173 (+    69us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:15:49.079232 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.693152 (+613920us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b540011e4d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:49.693199 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.693325 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32703,"prepare.queue_time_us":138472,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":11502,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":81771776,"thread_start_us":451325,"threads_started":2,"wal-append.queue_time_us":175556}
W20260709 12:15:49.695170 29783 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:15:49.699746 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.701745 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:49.702705 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.701913 23461 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:49.707361 29366 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:49.707476 29318 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:49.707968 29366 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.710290 23436 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:49.716157 23463 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:49.709278 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:15:49.718406 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.721086 29789 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:49.715852 29318 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.724450 29318 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.723440 29789 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.729506 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:49.725059 29781 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:15:49.733017 29781 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:49.733415 29781 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.734284 29781 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.735574 29781 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:49.722507 29442 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:49.738699 29680 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:49.739868 29740 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:49.740267 29740 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.741209 29740 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.741931 29674 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.742657 29748 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:49.742533 29674 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.744565 29789 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.743036 29721 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.744997 29674 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.745837 29789 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:49.742532 29740 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:49.746954 29559 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:49.747402 29559 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.747908 29598 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.745468 29567 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:49.756084 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.758172 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.762216 23461 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:15:49.762622 23461 pending_rounds.cc:85] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 13
I20260709 12:15:49.762893 23461 pending_rounds.cc:107] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:15:49.763161 23461 raft_consensus.cc:2889] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:49.751416 29651 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:49.753659 29674 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:49.784504 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.758291 29791 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.732102 29767 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:49.765076 29720 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:49.758028 29442 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:49.765342 29412 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:15:49.805564 29366 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.806118 29442 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.804180 29676 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:15:49.748364 29559 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.787132 23387 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:49.765591 29694 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.791452 23389 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:49.731287 29777 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:15:49.812937 29767 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.813810 29777 consensus_queue.cc:1243] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 54640ab8abeb4c949426d66c7e5aed87 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:15:49.813237 29694 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.815218 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
W20260709 12:15:49.815547 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.815428 29767 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.817029 29727 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:49.817178 29694 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.817662 29727 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:49.818589 29727 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.819810 23315 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:49.821311 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.759092 29399 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.822177 29399 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.824561 29399 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.803429 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.748399 29598 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.826527 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.827133 23466 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:49.827782 29598 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.827874 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.828469 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60318 (request call id 1334) took 1581 ms. Trace:
I20260709 12:15:49.828853 23470 rpcz_store.cc:276] 0709 12:15:48.246537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.246656 (+   119us) service_pool.cc:224] Handling call
0709 12:15:49.828270 (+1581614us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54003edfd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:49.828383 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993047424}
I20260709 12:15:49.830668 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:49.833220 29643 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:49.829846 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:49.838961 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:49.834712 29697 consensus_meta.cc:287] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.003s
I20260709 12:15:49.797704 29775 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.844300 23389 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:15:49.847878 23391 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:49.852591 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.853112 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.852456 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.864296 29488 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.864892 29488 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.867100 29488 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.868400 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.869228 23314 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:49.869865 29441 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.852679 29399 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:49.870820 29399 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:49.871258 29399 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:49.872666 29441 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.874842 29775 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.877029 29775 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.877864 29479 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:49.878898 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:49.880651 29399 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.852770 29725 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:15:49.850783 29764 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 12:15:49.882750 29704 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.006s
I20260709 12:15:49.883176 29704 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.802495 29791 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.885901 29791 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.886292 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.841638 29561 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:49.833559 29707 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:15:49.887881 29561 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:49.888664 29707 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:49.889035 29707 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:49.889565 29561 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.889943 29707 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.891273 29707 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:49.852375 29594 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:15:49.897367 29594 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:49.897899 29594 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.900708 29594 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.828462 29720 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.885182 29704 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:49.908394 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:49.911921 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43600.5us
I20260709 12:15:49.911238 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:49.841171 29767 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:49.836812 29643 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.917763 24016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 1056) took 1335 ms. Trace:
I20260709 12:15:49.882781 29399 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.920266 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.926299 24011 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:15:49.915673 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:49.928653 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.930164 24009 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:49.946030 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.909150 29789 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:49.927947 24016 rpcz_store.cc:276] 0709 12:15:48.581908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:48.582266 (+   358us) service_pool.cc:224] Handling call
0709 12:15:48.582577 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:48.582801 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:15:48.590996 (+  8195us) log.cc:844] Serialized 14 byte log entry
0709 12:15:49.723634 (+1132638us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b18003d59a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a921423a 00007ff0a967ea2a
0709 12:15:49.917217 (+193583us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:49.917300 (+    83us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:49.917374 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:49.917543 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:49.917695 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7156,"prepare.queue_time_us":9018,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":109,"raft.queue_time_us":12484,"spinlock_wait_cycles":168103168,"thread_start_us":185775,"threads_started":1}
I20260709 12:15:49.946836 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:49.937825 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 155160us
I20260709 12:15:49.948936 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.950111 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:49.949904 29789 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:49.907294 29599 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:49.901404 29594 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:49.973291 23312 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:15:49.924101 29437 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:50.013827 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 886653us
I20260709 12:15:49.839823 29580 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.979821 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:49.927655 29775 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:15:49.982048 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:49.983949 29789 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:49.984671 29594 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:49.985625 29725 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:49.985862 29313 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:49.988062 29764 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:49.990464 29720 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:49.990801 29655 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:15:49.916854 29488 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.003587 29643 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.009456 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.071696 23464 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:50.071913 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:49.930717 29704 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:50.017417 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:50.020334 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:50.077060 24008 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:50.025954 29674 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:50.035588 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:50.038525 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:50.081184 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:49.978020 29600 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.042465 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:49.971060 29781 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 12:15:50.061300 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:50.066676 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 158363us
I20260709 12:15:50.070475 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.075606 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:50.041210 29775 consensus_queue.cc:1243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:15:50.079604 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.087270 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.090034 23392 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:50.095240 24007 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:50.094544 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:50.074914 29720 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.101516 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.120932 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.122767 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:50.136734 23391 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:15:50.146543 23392 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:50.147203 23390 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:15:50.078763 29488 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.124818 29724 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:15:50.120893 29674 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:50.118057 29704 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:50.155352 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:50.146833 29782 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.018s	sys 0.014s
I20260709 12:15:50.145893 29598 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.160593 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.158766 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.161372 23393 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:50.121948 29366 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:50.167220 23318 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:50.149606 29781 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.120726 29720 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.093626 29600 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.179029 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.119714 29677 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
W20260709 12:15:50.168828 29782 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.019s	sys 0.014s
I20260709 12:15:50.189756 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:50.193886 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.196074 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.201175 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.203908 23392 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:50.209486 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.212761 24007 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:50.213138 29770 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.220141 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.223098 23468 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:15:50.180034 29789 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:15:50.206074 29567 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.194756 29782 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:50.255178 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.226089 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.256811 23317 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:50.257099 23318 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:50.179724 29598 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.214174 29643 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:50.268887 23317 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:15:50.265894 29800 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:15:50.233166 29733 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:50.233024 29580 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.213768 29770 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.274578 23314 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
W20260709 12:15:50.289084 29798 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:15:50.229014 29805 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.136s
I20260709 12:15:50.298945 29813 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.295853 29740 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.282373 29437 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.228724 29680 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:15:50.228915 29771 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:15:50.307262 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 97815us
I20260709 12:15:50.283413 29733 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:50.293776 29643 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.309335 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.311952 29801 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.342630 29801 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.329113 29643 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:50.328282 29808 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:15:50.346067 29643 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.350098 29720 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.352735 29488 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.313520 29721 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:15:50.353761 29782 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.356894 29598 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.358933 29600 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.358826 29733 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.359794 29594 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.360142 29733 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:50.352819 29807 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:50.360795 29807 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:50.361704 29807 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:50.362530 29798 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:15:50.367133 23436 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:50.362846 29807 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:50.370913 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:50.372308 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 63097.2us
I20260709 12:15:50.371758 23395 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:50.373912 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:50.381011 23387 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
W20260709 12:15:50.421299 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.425026 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:50.425772 23464 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:50.425875 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:50.427584 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50516.2us
I20260709 12:15:50.372835 29680 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:50.376359 29643 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.420708 29482 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.306406 29740 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.308985 29437 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.381932 29594 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:50.432868 29482 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.438549 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.451115 23464 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:50.451364 29770 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.452229 29482 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.452215 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.457571 29598 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:50.458230 29672 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:50.459052 29399 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:15:50.461799 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.461953 29805 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.462544 29805 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.464643 29805 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.464848 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.465176 29805 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.465667 29805 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.466048 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.467550 29805 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.473659 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 2175) took 1096 ms. Trace:
I20260709 12:15:50.429680 29462 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:50.473827 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:50.474128 23316 rpcz_store.cc:276] 0709 12:15:49.377314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.377813 (+   499us) service_pool.cc:224] Handling call
0709 12:15:50.473503 (+1095690us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400149454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:50.473604 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459654400}
I20260709 12:15:50.429793 29600 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.306543 29813 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.488015 23394 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:50.488065 23392 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:15:50.448701 29594 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.490571 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:50.491654 23395 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:50.479159 29462 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:50.493481 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.494287 23393 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:50.481201 29488 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.499584 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.500579 24011 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:50.500783 29807 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.502213 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.517001 23395 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:15:50.480948 29464 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:15:50.499145 29594 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:50.493213 29462 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:50.485020 29598 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:15:50.527825 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.501247 29807 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.527271 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:50.487111 29600 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.532793 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.538084 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.538890 23319 raft_consensus.cc:2410] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:15:50.533819 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.532866 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.506052 29733 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.532423 23316 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:50.541239 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.543051 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:50.537470 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.545086 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.544116 23312 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:50.532128 23469 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.546142 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.546898 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.547880 23315 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:50.551458 23312 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:50.551644 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.552121 23469 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:50.499287 29488 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.537710 29399 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:50.574523 29437 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.546838 29782 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.575784 29782 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.536685 29704 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.559248 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.577736 29704 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.578061 29782 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.578583 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:50.579130 29824 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:15:50.579460 23470 raft_consensus.cc:2410] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:15:50.579761 29704 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.529603 29811 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:15:50.537002 29672 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:15:50.580278 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.536758 29821 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:50.582975 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.583563 29807 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.549662 29733 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.585939 29733 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.585286 23462 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:50.571895 29813 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.573276 29740 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.570530 23321 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:15:50.581099 23470 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:15:50.549134 29580 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:50.592463 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.546185 29819 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:50.552601 29812 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:50.555757 29724 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.597781 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.588176 29823 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:50.598913 23462 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:15:50.607745 23462 pending_rounds.cc:85] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 13
I20260709 12:15:50.608099 23462 pending_rounds.cc:107] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:15:50.608448 23462 raft_consensus.cc:2889] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:50.598438 29580 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.606734 29819 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:50.619213 29819 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.607018 29823 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.622437 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.623780 23462 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:15:50.623541 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2758) took 1463 ms. Trace:
I20260709 12:15:50.627467 29464 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:50.624799 23360 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.609907 29812 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:50.631543 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.624737 23469 rpcz_store.cc:276] 0709 12:15:49.160105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.179917 (+ 19812us) service_pool.cc:224] Handling call
0709 12:15:49.193229 (+ 13312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:50.525611 (+1332382us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54002be920. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95f206f 00007ff0a966af60 00007ff0a966c7de 00007ff0a9676b49 00007ff0a9677416 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62
0709 12:15:50.623221 (+ 97610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.623262 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:50.623299 (+    37us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:50.623339 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.623398 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:50.623494 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"spinlock_wait_cycles":140039936}
I20260709 12:15:50.632020 23462 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:50.617597 29580 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:50.637336 23462 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:50.650354 23360 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:50.654999 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 117300us
I20260709 12:15:50.655055 29832 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:50.640957 29482 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:50.658136 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.660625 24009 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:50.608314 29724 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.663182 29724 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.663981 23313 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:50.665311 29832 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.666054 29600 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.665889 29832 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.652009 29567 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:50.631816 29805 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:15:50.624127 29823 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:50.671514 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.674292 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.649734 29817 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:50.649125 29704 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:50.628509 29464 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:15:50.675995 29567 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:50.676132 29832 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:50.682713 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 135882us
I20260709 12:15:50.678720 29817 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:50.694962 29827 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:15:50.697263 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:50.689255 29805 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:50.722193 23463 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2938) took 1517 ms. Trace:
I20260709 12:15:50.722759 23463 rpcz_store.cc:276] 0709 12:15:49.205692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.228644 (+ 22952us) service_pool.cc:224] Handling call
0709 12:15:49.231643 (+  2999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:49.716568 (+484925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:50.721849 (+1005281us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540044b954. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:50.722112 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113846144}
I20260709 12:15:50.723531 29817 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.724961 29817 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:50.687831 29812 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.732514 29733 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:50.735672 23320 raft_consensus.cc:1275] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:15:50.736654 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.737917 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.739528 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.740809 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.748414 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.735496 29805 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.717285 29488 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.730405 29594 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:50.747355 29784 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 1.138s	user 0.005s	sys 0.000s
W20260709 12:15:50.761747 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 87544.6us
I20260709 12:15:50.763845 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.764462 24009 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:15:50.767136 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.770334 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.754017 29811 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:15:50.780714 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.755127 29672 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:50.755962 29805 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:50.758632 29559 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:50.758170 29598 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:15:50.791882 29821 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:50.794790 29733 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.798918 29811 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:50.802819 29821 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.831254 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.832562 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:50.835003 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:50.835033 23391 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:15:50.841760 23391 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:50.842294 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.842507 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.842710 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.803021 29462 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.843160 23396 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:50.843331 23317 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
W20260709 12:15:50.828429 29834 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:15:50.849691 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.847064 29748 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:50.845047 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.839409 29821 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:50.806145 29733 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:50.858625 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.862954 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.841650 29807 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:50.862154 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:50.867416 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:50.884546 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:50.892629 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:50.893096 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 138159us
W20260709 12:15:50.891606 29831 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:15:50.851913 29748 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:50.901032 29851 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:50.902757 29839 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:15:50.864185 29838 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.868229 29740 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:50.865684 29837 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.909605 29674 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:15:50.914186 29851 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.913251 23463 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:50.919070 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:50.921021 29674 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:50.921408 29674 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:50.898969 29807 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.898883 29770 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:50.928367 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:50.929629 29737 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:50.930202 29737 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:50.930430 29845 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:50.930600 29737 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:50.932415 29733 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.926942 29767 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:50.924350 29594 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:50.933403 29767 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:50.896762 29782 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:15:50.941522 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:50.914444 29740 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:50.945892 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:50.947492 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.933965 29770 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:50.956910 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:50.939313 29848 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:15:50.948608 29740 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:50.958137 29462 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:50.912199 29811 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.919030 29838 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:50.996574 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 59914.5us
I20260709 12:15:50.934937 29807 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.964638 29770 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:50.925770 29837 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.967669 29851 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:50.954408 29202 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:15:50.978647 29600 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:51.017803 29462 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.034725 29821 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.042925 29462 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.045060 29462 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.046965 23394 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 22: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:51.050824 29837 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.038004 29600 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.053606 29600 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.055596 29767 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.038465 29842 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:15:51.059510 23465 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:51.059373 29805 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.060549 23395 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:51.064666 29823 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:50.962076 29748 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.072490 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.049923 29821 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.072867 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.072244 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.075302 23320 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:51.067811 29801 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.100275 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:51.104019 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:51.108119 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 105121us
I20260709 12:15:51.102725 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.117786 23360 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:51.121964 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.128324 29707 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:51.128861 29707 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:51.129886 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.129823 29707 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.130861 29707 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:51.134125 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.109730 29677 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:15:51.118718 29561 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:51.139081 29838 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.143529 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.153537 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:51.123754 29821 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:15:51.155084 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.158219 24009 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:15:51.149319 29561 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.175153 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.133944 29868 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:15:51.162032 29838 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:51.200270 23467 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:51.202020 23467 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:51.175601 29807 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:51.209227 29872 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:15:51.209969 29805 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.210270 29870 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:51.201960 29838 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.211563 23467 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.161468 29580 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.228487 29870 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:51.234090 29580 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.233912 29870 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.234853 24008 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:51.235280 29870 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.232228 29823 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.238751 29740 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.240551 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.240859 29740 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.179615 29561 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.241833 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.242648 29674 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.243762 29674 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:51.245620 29838 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:51.247900 29561 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.251173 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.251780 23312 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.259696 23467 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.279891 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2177) took 1245 ms. Trace:
I20260709 12:15:51.280357 23315 rpcz_store.cc:276] 0709 12:15:50.034716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.034860 (+   144us) service_pool.cc:224] Handling call
0709 12:15:50.070513 (+ 35653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:50.550088 (+479575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.279636 (+729548us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54000ecc54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:51.279825 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097426176}
I20260709 12:15:51.281978 23312 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:51.282644 29561 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.284256 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.285148 29561 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:51.286180 29561 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:51.287058 29561 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.285622 29642 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:51.289984 29642 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.291981 29642 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.300601 23396 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: []
W20260709 12:15:51.318043 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:51.317817 29817 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.592s	user 0.013s	sys 0.000s
I20260709 12:15:51.331581 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.320915 23467 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:51.337383 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.349988 23391 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:51.350132 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1345) took 1651 ms. Trace:
I20260709 12:15:51.350800 23467 rpcz_store.cc:276] 0709 12:15:49.698617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.735681 (+ 37064us) service_pool.cc:224] Handling call
0709 12:15:49.735945 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:51.200141 (+1464196us) spinlock_profiling.cc:243] Waited 85.6 ms on lock 0x7b540017f320. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ebdc0 00007ff0a96761c0 00007ff0a9677231 00007ff0a9673d8f 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c
0709 12:15:51.349790 (+149649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.350067 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196959104}
W20260709 12:15:51.338114 23465 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2769) took 1434 ms (client timeout 1682 ms). Trace:
W20260709 12:15:51.352900 23465 rpcz_store.cc:269] 0709 12:15:49.904195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.928393 (+ 24198us) service_pool.cc:224] Handling call
0709 12:15:51.337954 (+1409561us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003db154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:51.338056 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559946624}
I20260709 12:15:51.344933 23393 raft_consensus.cc:1240] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:51.347378 23394 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:51.362905 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.366798 29844 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.367509 29844 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.376540 29740 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.377132 29740 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.379213 29740 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.380342 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1117) took 1222 ms. Trace:
I20260709 12:15:51.380839 23390 rpcz_store.cc:276] 0709 12:15:50.157651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:50.157791 (+   140us) service_pool.cc:224] Handling call
0709 12:15:50.158076 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:50.175811 (+ 17735us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:50.175842 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:50.175884 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:15:50.175924 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.380034 (+1204110us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b54002e9854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:51.380094 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.380268 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40247680}
I20260709 12:15:51.389091 29740 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:51.389737 29740 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.390800 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.391713 29740 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:51.359436 29652 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:15:51.356314 29868 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.400132 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.409157 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.410548 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.420113 23466 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:51.411566 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.411948 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:51.412784 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.420002 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:51.320381 29770 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.421620 24009 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:51.420249 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:51.436930 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.445641 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.447676 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:51.423149 29860 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:15:51.423005 29831 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:15:51.452234 29801 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:15:51.414997 29823 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:51.418639 29725 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:51.455659 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:51.448617 29737 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.001s
I20260709 12:15:51.448673 29740 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.470994 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.472055 29823 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.468228 29643 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:51.475138 29643 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.465626 29366 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:51.475963 29366 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:51.476008 29823 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.476477 29366 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:51.469919 29870 consensus_queue.cc:1059] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 12:15:51.477638 29870 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:51.477975 29870 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:51.470118 29202 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.478742 29870 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.480027 29870 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:51.481782 29811 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:51.485819 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2946) took 1601 ms. Trace:
I20260709 12:15:51.486235 23388 rpcz_store.cc:276] 0709 12:15:49.884119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:49.884281 (+   162us) service_pool.cc:224] Handling call
0709 12:15:49.884698 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:49.885053 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:49.885080 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:51.320947 (+1435867us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b2c0005c228. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:51.323803 (+  2856us) log.cc:844] Serialized 26 byte log entry
0709 12:15:51.468028 (+144225us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:51.468068 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:51.468142 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:51.485529 (+ 17387us) raft_consensus.cc:1685] finished
0709 12:15:51.485547 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:51.485757 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":361,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":51066624,"thread_start_us":110103,"threads_started":1,"wal-append.queue_time_us":138647}
I20260709 12:15:51.488682 29823 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.474152 29740 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.469300 29318 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:15:51.507323 29740 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.507388 29779 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.518115 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.521307 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:51.524120 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.524578 23462 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:51.524752 23390 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:51.527302 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.530083 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.530725 23393 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:51.420912 29594 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:15:51.517395 29823 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.548902 29823 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.558909 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.471719 29737 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.490182 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:51.423499 29724 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.110s
W20260709 12:15:51.568737 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 78669.5us
I20260709 12:15:51.565598 23394 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:51.566080 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:51.496973 24009 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:51.570065 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.574988 23391 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:51.587730 23470 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:15:51.531585 29844 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.590413 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.540874 29366 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.592870 23313 raft_consensus.cc:2393] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:51.551326 29643 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.597775 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.602520 23392 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:51.492753 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:51.595180 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.603084 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.606362 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.606983 23316 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:51.607824 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.594009 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:15:51.605877 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.604480 23463 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:51.599005 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:51.605876 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.611821 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.612514 23393 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:51.614161 23980 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:51.615257 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.616250 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:51.617631 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 124995us
I20260709 12:15:51.619369 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.614048 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:51.587009 29399 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:51.620000 23315 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:51.624909 23468 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:15:51.627087 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.635819 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.636584 23468 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:51.640444 23390 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:15:51.643716 29437 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:51.644192 29437 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.644546 29437 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:51.620354 29643 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:51.625979 29652 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:51.658396 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:51.662058 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.660334 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:51.664003 29740 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:51.623080 29821 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
W20260709 12:15:51.666977 29845 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 12:15:51.667388 29845 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:51.661309 29652 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.669021 29652 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:51.669502 29807 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:51.648447 29677 consensus_queue.cc:1048] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:15:51.682510 29823 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:51.682956 29823 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:51.683158 29737 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.683387 29823 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:51.674685 29789 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:15:51.683172 29740 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:51.685964 29740 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.686381 29740 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:51.657034 29779 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:15:51.688822 29779 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:51.694972 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.705473 29789 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:51.705947 29789 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.706305 29789 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:51.711153 23318 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:51.689632 29643 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:51.723827 29643 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.726678 29643 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.740806 23436 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:51.689215 29779 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:51.747896 24009 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:51.749949 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.751427 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.753000 24009 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:51.754571 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.758208 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.763391 29697 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.765300 29779 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.766633 29779 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:51.765566 29697 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:51.727424 29874 consensus_queue.cc:1175] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:15:51.779264 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:51.781833 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:51.756269 29771 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:51.786840 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.757225 29770 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.787707 23393 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:51.786840 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.791918 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:51.788803 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 130530us
I20260709 12:15:51.792908 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:51.789875 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.795082 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:51.781091 29781 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.785419 29697 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.797683 29599 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:51.803670 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 171876us
I20260709 12:15:51.794230 29771 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:51.803653 29767 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:51.802448 29781 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.814873 29767 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:51.814909 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:51.815644 23388 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:15:51.816887 29781 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.810914 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.798691 29770 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:51.822048 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.822737 23315 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:51.822999 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:51.821237 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.807307 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:51.815435 29767 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:51.813246 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:51.824738 23320 raft_consensus.cc:2368] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16.
I20260709 12:15:51.829082 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:51.833011 29789 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.835913 29789 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.825914 29781 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:51.827523 29870 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:15:51.841672 23388 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
W20260709 12:15:51.845463 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:51.818189 29811 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:15:51.849514 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:51.852387 23391 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:51.847411 29599 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.852874 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:51.854456 23462 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:51.855628 29674 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:51.856591 29892 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:15:51.860330 29720 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:51.885514 23316 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:15:51.863026 29889 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.206s	user 0.002s	sys 0.000s
I20260709 12:15:51.863551 29437 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.863950 29697 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:51.848022 29313 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:51.864707 29895 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:51.907361 29895 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.907776 29895 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.864976 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:51.865509 29890 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:15:51.866271 29771 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:51.842621 29838 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.001s
I20260709 12:15:51.878156 29807 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:51.881922 23320 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:51.886108 29594 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:51.882968 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:51.814185 29652 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.888831 29720 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.891112 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.891057 29724 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:51.891522 29313 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.893029 29746 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:51.906314 29805 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:51.848692 29823 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.933094 29823 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:51.925015 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 60133.5us
I20260709 12:15:51.925112 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:51.926157 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:51.929574 29893 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:51.929704 29897 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:51.929586 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:51.930102 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 90195.2us
I20260709 12:15:51.943143 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.942672 23392 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:51.931054 29313 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:51.954674 29561 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:51.957054 29720 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.958861 29652 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.921525 29838 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:51.960595 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:51.941920 29737 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:51.962666 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:51.963353 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:51.963518 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.963840 23395 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:51.942939 24011 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:51.939638 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:51.940618 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:51.966392 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.966962 23315 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:51.967178 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:51.934727 29437 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:51.953050 29893 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:51.962718 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.918917 29896 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:51.915141 29807 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:51.950212 23320 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:51.973903 23314 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:51.974781 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:51.961390 29838 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.985567 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:52.000496 23434 leader_election.cc:400] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16.
I20260709 12:15:52.001085 23434 leader_election.cc:403] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:15:51.936744 29746 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:51.975060 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:52.001909 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57099.4us
W20260709 12:15:52.003063 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54515.4us
I20260709 12:15:51.936580 29870 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:51.996165 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:51.996733 29902 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:15:52.005132 23461 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:52.001312 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:52.025992 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 51040.5us
I20260709 12:15:52.028122 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.029224 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:52.031236 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:52.031831 23391 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:52.045267 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:52.045635 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:52.046060 23396 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:51.938328 29899 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:51.961383 29561 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:52.052472 23396 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:52.057118 23314 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:52.063088 23391 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:51.961460 29737 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:51.987720 29694 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:52.068110 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44707.4us
I20260709 12:15:51.984531 29697 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:52.048524 29599 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:52.078040 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53472.8us
I20260709 12:15:52.085785 23468 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:52.087179 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:52.073174 29838 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:52.044525 29789 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:52.071322 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44713.8us
I20260709 12:15:52.022985 29805 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:52.117025 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.131870 23387 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:15:52.132313 23387 pending_rounds.cc:85] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 13
I20260709 12:15:52.132547 23387 pending_rounds.cc:107] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:15:52.132786 23387 raft_consensus.cc:2889] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:52.075763 29870 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:52.022320 29897 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:52.141506 29890 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:15:51.989542 29842 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:15:52.143364 29842 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:52.143716 29842 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:52.144554 29842 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.126690 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:52.148025 29891 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.346s	user 0.001s	sys 0.000s
I20260709 12:15:52.149893 29692 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:52.150327 29692 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.150694 29692 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:52.153862 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:52.155493 29770 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:52.136341 29805 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.126183 29652 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:52.141258 29580 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:52.125572 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.114198 29694 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.158701 29838 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:52.178519 29771 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:52.224277 29748 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:52.190821 29694 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:52.232161 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.235211 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.236020 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:52.236285 24010 raft_consensus.cc:1240] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:52.225143 29655 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.201654 29652 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:52.269397 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:52.183712 29740 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.276705 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.287181 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.289620 29830 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:52.125707 29746 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.313480 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:52.315168 29830 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.315876 29830 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.322918 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.334669 23391 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:52.335975 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:52.335346 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 175471us
I20260709 12:15:52.215024 29779 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:15:52.346426 23461 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:52.340550 23362 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.356046 23462 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:52.358165 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.365751 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:52.368593 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:52.346877 29594 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:52.249161 29737 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.307639 29823 consensus_queue.cc:1059] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:15:52.278424 29655 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.363029 29897 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:52.349858 29770 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:52.351008 29899 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:52.281621 29870 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:52.366415 29767 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.001s
I20260709 12:15:52.332849 29847 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:15:52.373822 29721 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:52.367458 29488 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:52.404470 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.333089 29812 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:15:52.391032 29596 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:52.394671 29893 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:52.419071 23315 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:52.406612 29707 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:52.366606 29838 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.401006 29899 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.410130 29721 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:52.386647 29652 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.449553 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:52.413070 29823 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:52.399762 29767 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.424810 29779 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:52.456722 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.467698 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.468947 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:52.429776 29707 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:52.423245 29596 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.451663 29805 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:52.457583 29903 log.cc:927] Time spent T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:15:52.498046 23393 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
W20260709 12:15:52.494755 29900 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 12:15:52.496031 29906 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:15:52.464926 29770 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.497295 29897 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:52.548365 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 80750.7us
I20260709 12:15:52.527895 29740 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:52.538990 29905 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.188s	user 0.000s	sys 0.002s
W20260709 12:15:52.554899 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.541661 29915 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.542133 29697 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:52.551019 29908 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:15:52.542223 29366 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.562920 23312 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:15:52.554708 29907 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:15:52.474725 29823 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.493502 29779 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.553174 29922 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:52.545977 29707 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.584852 29655 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.548738 29721 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:52.643989 29918 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:15:52.656956 29924 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:15:52.637612 29897 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:52.644889 29805 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:52.647204 29917 log.cc:927] Time spent T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:15:52.564476 29697 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.739113 29697 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:52.741576 29921 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:15:52.743014 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:52.728554 29442 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.746017 29442 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.748085 29442 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:52.745226 29692 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
I20260709 12:15:52.755066 29692 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.561548 29915 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:52.771211 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:52.780886 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.777113 29767 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.777226 29805 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.697062 29596 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.652786 29830 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.814603 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:52.804087 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.806231 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.816368 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:52.817701 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.812085 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:52.809338 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:52.820470 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:52.819345 23319 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:52.831192 23470 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:52.849759 29946 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:52.733529 29707 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:52.815162 29895 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.907s	user 0.013s	sys 0.000s
I20260709 12:15:52.761909 29930 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.850575 29946 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:52.775728 29915 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:52.761390 29932 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.799893 29594 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.847399 29674 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.770710 29692 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.829587 29442 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:52.888307 23467 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.020s	sys 0.035s
W20260709 12:15:52.888676 23467 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.020s	sys 0.035s
W20260709 12:15:52.884682 29599 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.759s	user 0.018s	sys 0.000s
I20260709 12:15:52.890705 29599 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16.
I20260709 12:15:52.891968 23468 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:15:52.892768 29932 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.893903 29442 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.895120 29932 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:52.885532 29748 consensus_meta.cc:287] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.001s
I20260709 12:15:52.910398 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:52.918542 23465 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:52.911808 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:52.916326 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:52.907724 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:52.916090 29737 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:52.920404 24008 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 22.
I20260709 12:15:52.925541 23321 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:52.926756 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.875687 29817 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:15:52.927099 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:52.886889 29946 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:52.937582 29913 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:52.954507 29913 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:52.955076 29913 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:52.954823 29946 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:52.955935 29927 log.cc:927] Time spent T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.111s	user 0.000s	sys 0.002s
I20260709 12:15:52.958554 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:52.959223 23469 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:52.959802 29442 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:52.937456 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:52.962059 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:52.962945 29694 consensus_meta.cc:287] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.002s
I20260709 12:15:52.934839 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:52.941864 29823 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.943111 29805 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:52.888836 29594 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.888397 29930 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:52.950214 29807 consensus_meta.cc:287] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.000s
I20260709 12:15:52.966023 29930 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.966038 23396 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 22.
I20260709 12:15:52.967644 29594 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:52.888551 29674 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.972437 29674 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.920724 29642 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:52.966362 29807 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.974032 29642 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.976155 29807 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:52.976753 29642 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:52.983492 23468 raft_consensus.cc:2368] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16.
I20260709 12:15:52.986325 23469 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:52.951465 29779 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:52.880666 29895 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.780323 29897 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:52.927801 29655 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:52.929911 29697 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:15:52.963367 29694 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:52.949115 29922 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:15:53.001103 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 180743us
W20260709 12:15:53.002349 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 187853us
I20260709 12:15:53.039788 23393 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:52.982815 29202 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.055171 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.057741 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.058421 24008 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:15:52.934100 29737 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.060932 29202 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.071002 29737 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.072896 29202 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:52.999475 29655 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.071087 29694 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.055876 29913 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.073276 29897 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.083748 29949 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.093448 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.093837 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.094353 23312 raft_consensus.cc:2368] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:53.094583 23314 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:53.086122 29807 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.089603 29949 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.113862 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.112823 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.119839 23469 raft_consensus.cc:2368] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:53.119870 23320 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
W20260709 12:15:53.121544 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 206415us
I20260709 12:15:53.165297 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.177748 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:53.114741 29807 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.224661 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.229552 29949 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:53.220978 29907 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:15:53.223663 29441 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:53.231403 29441 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.231798 29441 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:53.232461 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.219844 29596 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:15:53.235550 29950 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:53.235981 29950 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:53.236804 29950 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.231410 29807 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.247797 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.248661 29674 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:53.249157 29674 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.249533 29674 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:53.250614 29953 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.251153 29953 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:53.253947 29938 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:15:53.254793 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.255671 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2794) took 1021 ms. Trace:
I20260709 12:15:53.255864 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2973) took 1294 ms. Trace:
I20260709 12:15:53.257014 23393 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:53.256053 23466 rpcz_store.cc:276] 0709 12:15:52.234555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.234685 (+   130us) service_pool.cc:224] Handling call
0709 12:15:52.234973 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.235292 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.235322 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.768247 (+532925us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:52.768724 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:15:52.886534 (+117810us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:52.921973 (+ 35439us) log.cc:844] Serialized 14 byte log entry
0709 12:15:52.922220 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:52.922313 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.255389 (+333076us) raft_consensus.cc:1685] finished
0709 12:15:53.255485 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.255615 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41275,"prepare.queue_time_us":39432,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":17432,"raft.queue_time_us":147431,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":7806,"spinlock_wait_cycles":12304256,"thread_start_us":117402,"threads_started":1,"wal-append.queue_time_us":236229}
I20260709 12:15:53.256254 23389 rpcz_store.cc:276] 0709 12:15:51.961931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.964024 (+  2093us) service_pool.cc:224] Handling call
0709 12:15:51.976126 (+ 12102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.091835 (+115709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.091865 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:52.091910 (+    45us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:52.091938 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.255646 (+1163708us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540036cc54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:53.255695 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.255815 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265644928}
I20260709 12:15:53.258358 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:53.259099 29942 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:15:53.262712 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.263528 23362 leader_election.cc:400] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:53.263880 23362 leader_election.cc:403] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:53.264369 29674 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.266165 29721 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:53.266984 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.265198 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.267023 29838 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.268471 29724 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:53.269017 29724 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:53.269759 29674 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:53.271065 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.271677 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:53.273728 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.273880 23470 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:15:53.275576 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.276302 23469 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:53.277036 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:53.281066 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:53.261287 29441 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.269896 29724 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.280185 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.284493 24010 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:53.270301 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2790) took 1037 ms. Trace:
I20260709 12:15:53.282490 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.285681 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.286159 23320 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:53.287075 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.287616 23466 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:53.290138 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.290381 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.290908 23387 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:53.293234 23394 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:15:53.293882 23393 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:53.283774 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:53.290153 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.294340 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.294803 23389 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:53.285257 23461 rpcz_store.cc:276] 0709 12:15:52.233067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.249375 (+ 16308us) service_pool.cc:224] Handling call
0709 12:15:53.270089 (+1020714us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b54002265d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:53.270244 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111153024}
I20260709 12:15:53.299270 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.300765 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:53.299188 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 188011us
I20260709 12:15:53.304912 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.306915 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.307080 29652 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:53.307245 29697 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:53.307530 23320 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:53.307991 29652 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:53.309044 24010 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:15:53.309725 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.279948 29838 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:15:53.318642 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.253351 29953 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.311293 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.314916 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.316527 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:53.324556 23318 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:53.325850 23469 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:53.328612 23392 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:53.329490 29781 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.330040 23464 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
W20260709 12:15:53.330472 29892 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.163s	user 0.010s	sys 0.003s
W20260709 12:15:53.330824 29892 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.010s	sys 0.003s
I20260709 12:15:53.330084 29781 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.333611 29781 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.337401 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:53.338119 23468 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:53.338591 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.339458 23395 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:53.341747 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.342236 23434 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.342464 24008 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:53.343672 29771 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:53.299773 29895 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:53.304442 29707 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:53.304206 29897 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:53.330837 23387 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:53.361819 23318 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:53.362092 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.368892 23287 leader_election.cc:400] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16.
I20260709 12:15:53.354719 29781 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.371515 29914 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.373458 29781 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.374248 29914 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.374922 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2795) took 1140 ms. Trace:
I20260709 12:15:53.376377 29914 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.375291 23467 rpcz_store.cc:276] 0709 12:15:52.234822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.278909 (+ 44087us) service_pool.cc:224] Handling call
0709 12:15:52.279243 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.889095 (+609852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.889127 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:52.906087 (+ 16960us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.293269 (+387182us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:15:53.293305 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.293370 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:53.374711 (+ 81341us) raft_consensus.cc:1685] finished
0709 12:15:53.374731 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.374864 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122292,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":4739,"thread_start_us":904998,"threads_started":2,"wal-append.queue_time_us":440457}
W20260709 12:15:53.362535 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51312.7us
I20260709 12:15:53.372428 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.380793 29771 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:53.381317 29771 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:53.382141 29771 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.364435 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.383991 29915 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:53.355067 29707 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.376281 29781 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.385823 29707 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:53.385967 29842 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:53.386965 23362 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.386986 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.382534 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2980) took 1219 ms. Trace:
I20260709 12:15:53.370838 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.389868 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.389487 23391 rpcz_store.cc:276] 0709 12:15:52.163626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.190515 (+ 26889us) service_pool.cc:224] Handling call
0709 12:15:52.334450 (+143935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.335143 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.382342 (+1047199us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54002c8b50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:53.382475 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330095488}
I20260709 12:15:53.371163 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.393651 23320 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.389504 23468 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:15:53.405854 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.407678 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1151) took 1275 ms. Trace:
I20260709 12:15:53.408377 23390 rpcz_store.cc:276] 0709 12:15:52.132604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.132731 (+   127us) service_pool.cc:224] Handling call
0709 12:15:52.133028 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.206729 (+ 73701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.206772 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:52.206831 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:52.206883 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.407167 (+1200284us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b54002e8e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:53.407227 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:53.407598 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168879360}
I20260709 12:15:53.408744 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.412500 24011 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:53.414304 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.399863 29897 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:53.423974 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.435513 23320 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:53.437371 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.440243 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.417009 29737 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:53.416944 29945 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:53.424145 29913 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.456085 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.450747 29737 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:53.459319 29737 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:53.466073 29965 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:15:53.475035 29442 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:53.475904 29442 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.479348 29442 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:53.479913 29899 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.480517 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:53.484706 29964 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:15:53.487340 29746 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:53.454213 29922 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.497941 29922 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.500109 29922 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.502930 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.506876 29441 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.446707 29931 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.507923 29931 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.510004 29931 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.449978 29697 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:53.511495 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.513708 23316 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:53.512411 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.454563 29945 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.515829 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.516417 29961 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:53.516867 29961 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.517184 29961 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:53.518919 23315 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:53.522850 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2792) took 1289 ms. Trace:
I20260709 12:15:53.523239 23464 rpcz_store.cc:276] 0709 12:15:52.233858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.249498 (+ 15640us) service_pool.cc:224] Handling call
0709 12:15:52.249911 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.330533 (+1080622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.522617 (+192084us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:53.522792 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2483546368}
I20260709 12:15:53.515936 29945 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:53.524689 23360 leader_election.cc:400] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:53.527087 29913 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.527762 29913 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.534565 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:53.535983 29892 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:53.536371 29892 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:53.536725 29892 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:53.524207 29697 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:53.538566 29697 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.544185 29950 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:53.549854 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2201) took 1930 ms. Trace:
I20260709 12:15:53.550597 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.550557 29488 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:53.552162 29931 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:53.552609 29655 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:53.552366 29441 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:53.557139 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.526547 29849 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:15:53.558581 29441 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:53.558720 23317 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:53.560851 29561 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:53.561595 29561 consensus_queue.cc:1243] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:15:53.562152 29561 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:53.562539 29561 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:53.563516 29561 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.567158 23319 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:53.568856 29944 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.569873 29944 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:53.570845 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44677.7us
I20260709 12:15:53.572134 29845 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.523964 29707 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.572664 29845 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.574779 29849 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.575165 29707 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.575304 29849 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.575870 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:53.576678 23315 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 23: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:53.578632 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.550395 23321 rpcz_store.cc:276] 0709 12:15:51.619585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:51.619774 (+   189us) service_pool.cc:224] Handling call
0709 12:15:51.620072 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:52.925907 (+1305835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:53.549555 (+623648us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004236d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:53.549781 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3001579776}
I20260709 12:15:53.582715 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.583339 23317 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:53.585891 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.586889 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:53.587203 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:53.587908 24008 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:15:53.595338 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:53.565140 29561 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:53.598817 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.599932 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.602691 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.603973 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.605263 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.607237 23394 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:53.609225 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:53.611137 23391 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:53.611196 23317 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:53.572535 29944 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.613813 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.615003 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:53.618947 23391 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 23: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:53.616281 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.630225 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.631520 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.604225 29949 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:53.600711 29594 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.636018 24008 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 23.
I20260709 12:15:53.636180 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:53.638873 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.773694 23316 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:53.683604 29892 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.633214 29599 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:53.663686 29949 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.617970 29899 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:15:53.811563 29865 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:53.603914 29721 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.778395 29897 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:53.784197 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:53.799865 29838 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.815011 23393 raft_consensus.cc:1240] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:53.815171 29838 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.694281 29842 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:53.822804 29845 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.804106 29934 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:53.825268 29934 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.826349 29949 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:53.786448 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:53.794932 23391 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 12:15:53.829514 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 165572us
W20260709 12:15:53.797967 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 159169us
I20260709 12:15:53.831081 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.605150 29707 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:15:53.808296 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 487634us
I20260709 12:15:53.845399 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.865998 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:15:53.925940 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65433.8us
W20260709 12:15:53.933852 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102851us
I20260709 12:15:53.783260 29488 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:53.803264 29944 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.945665 23464 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:53.949808 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:53.827765 29934 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.843842 29313 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:53.942374 29789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.595s	user 0.017s	sys 0.039s
I20260709 12:15:53.926136 29943 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:15:53.828814 29849 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:53.973454 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:53.974072 23463 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:53.928272 29897 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:53.933559 29676 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:53.938109 29821 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.936726 29895 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:53.803779 29594 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.942655 29737 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:53.814462 29721 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.974596 29707 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:53.975644 29934 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:53.926561 29961 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.829558 29771 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.141s
I20260709 12:15:53.950876 29838 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:53.828728 29599 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:53.977667 29789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.018s	sys 0.039s
I20260709 12:15:53.990453 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.991905 29441 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:53.967736 29746 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:15:53.961822 29488 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.025835 23390 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:53.977926 29692 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:15:53.994696 29943 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:53.976557 29313 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:53.943435 29945 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.995676 29821 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.974900 29944 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.007603 29707 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:54.034643 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:53.995033 29897 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:53.995324 29895 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.051290 23464 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:15:54.082837 23464 pending_rounds.cc:85] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 14
I20260709 12:15:54.083220 23464 pending_rounds.cc:107] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:15:54.083520 23464 raft_consensus.cc:2889] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:54.035065 29737 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:54.034457 29892 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.073088 29599 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.033022 29899 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:54.083807 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49237us
I20260709 12:15:54.087733 23318 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:54.071081 29594 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:54.141826 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 336553us
I20260709 12:15:54.050328 29789 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:54.088972 23396 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:54.021837 29934 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.093402 29441 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:54.153132 29789 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:54.178200 29594 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.183804 29838 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:54.210104 29979 log.cc:927] Time spent T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:15:54.177783 29599 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:15:54.220651 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.164223 29899 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.229123 23387 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
W20260709 12:15:54.228016 29697 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.011s	sys 0.029s
I20260709 12:15:54.193717 29707 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.213028 29594 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.264067 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:54.268373 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:54.213692 29838 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.269338 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:54.260824 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 316038us
I20260709 12:15:54.276767 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:54.265065 29697 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.011s	sys 0.029s
I20260709 12:15:54.206609 29943 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.297995 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:54.291801 29599 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.301362 29944 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.302922 29771 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.303625 29771 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.304337 29897 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.305316 29821 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.302286 29599 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.306712 29945 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.306531 29949 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.309024 29599 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.316789 29838 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:54.318107 29642 consensus_queue.cc:1059] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:15:54.319507 29975 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:15:54.326639 29707 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:54.328799 29599 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.329608 29599 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.332922 29899 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.337898 29599 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.310074 29594 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.343490 29771 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.345453 29771 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.346186 29771 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.346776 29961 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:54.310151 29949 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:54.302611 29985 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.350307 29985 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:54.351241 29950 consensus_meta.cc:287] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.806s	user 0.012s	sys 0.004s
I20260709 12:15:54.278482 29868 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:54.302336 29561 consensus_meta.cc:287] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.000s
I20260709 12:15:54.353356 29985 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.353818 29868 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.356788 29934 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.357990 29868 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.360915 29771 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.369069 29721 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.431301 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:54.432135 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 134240us
I20260709 12:15:54.518082 23389 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:54.518760 23389 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:54.519791 23389 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.521018 23389 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:15:54.518030 23980 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.009s	sys 0.021s
I20260709 12:15:54.472970 29950 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:54.473131 29721 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:54.536446 24007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.003s	sys 0.023s
I20260709 12:15:54.472543 29899 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:54.472110 29771 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:54.539547 24007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.023s
I20260709 12:15:54.503995 29561 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.504410 29697 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:54.473373 29599 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:54.552906 23980 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.009s	sys 0.021s
I20260709 12:15:54.561270 23395 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:54.567500 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.568351 23395 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:15:54.569450 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:54.572330 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 619345us
W20260709 12:15:54.543932 29946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.008s	sys 0.013s
I20260709 12:15:54.543094 29721 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.631438 23395 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:54.599010 29707 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.553568 29561 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.639047 29707 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.640758 29561 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.549702 29771 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.641887 29707 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.643445 29771 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:54.644145 29771 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:54.552991 29899 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.647469 29899 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:54.649842 29721 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.557183 29697 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:54.653959 23435 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.007s	sys 0.020s
I20260709 12:15:54.644726 29771 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:54.627652 29946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.009s	sys 0.013s
I20260709 12:15:54.621713 29599 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.659885 24007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 1117) took 1068 ms. Trace:
I20260709 12:15:54.660364 24007 rpcz_store.cc:276] 0709 12:15:53.591880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.592026 (+   146us) service_pool.cc:224] Handling call
0709 12:15:53.592346 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.593884 (+  1538us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:15:53.946589 (+352705us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.540103 (+593514us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.540724 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 12:15:54.540896 (+   172us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:15:54.540943 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.541031 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.659642 (+118611us) raft_consensus.cc:1685] finished
0709 12:15:54.659663 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.659826 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1142,"prepare.queue_time_us":226427,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":6350,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":18095,"thread_start_us":945466,"threads_started":2,"wal-append.queue_time_us":711281}
I20260709 12:15:54.659648 29599 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.678725 29771 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:54.679708 29697 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:54.671607 29849 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.015s	sys 0.024s
W20260709 12:15:54.686350 29849 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.016s	sys 0.025s
W20260709 12:15:54.813344 23362 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.009s	sys 0.014s
I20260709 12:15:54.814368 29821 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.815946 29821 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.820418 29821 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:54.822098 23435 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.007s	sys 0.020s
W20260709 12:15:54.656569 23434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.006s	sys 0.020s
W20260709 12:15:54.814329 23362 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.009s	sys 0.015s
I20260709 12:15:54.847297 29697 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:54.852851 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.864370 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:54.881017 23434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.007s	sys 0.021s
I20260709 12:15:54.885380 29990 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:54.892442 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:54.893982 23320 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 20: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:54.894258 29997 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.876060 29821 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:15:54.899494 29996 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:15:54.900723 23462 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2797) took 2616 ms. Trace:
I20260709 12:15:54.884096 29989 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:54.902624 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:54.914882 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.917770 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.918709 23388 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:54.897214 29599 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.921785 29821 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.922554 29599 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.924880 29821 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:54.925855 29779 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.925908 29874 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:54.901228 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 2220) took 1530 ms. Trace:
I20260709 12:15:54.928766 29874 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:54.927314 23318 rpcz_store.cc:276] 0709 12:15:53.370422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.370553 (+   131us) service_pool.cc:224] Handling call
0709 12:15:54.900805 (+1530252us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400296154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:54.901023 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604126080}
I20260709 12:15:54.929359 29874 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:54.929769 29868 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:54.930750 29868 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.938922 29934 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:54.939889 29934 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.941103 29950 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:54.886577 29990 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.945885 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:54.947446 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2361) took 1166 ms. Trace:
I20260709 12:15:54.947870 23394 rpcz_store.cc:276] 0709 12:15:53.780996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.781224 (+   228us) service_pool.cc:224] Handling call
0709 12:15:53.799814 (+ 18590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:54.084942 (+285128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:54.085106 (+   164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:15:54.085175 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:54.085262 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.946871 (+861609us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b54002c8b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:54.946952 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.947243 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654919424}
I20260709 12:15:54.942375 29950 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.956738 29950 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:54.957938 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 2351) took 1692 ms. Trace:
I20260709 12:15:54.958421 23392 rpcz_store.cc:276] 0709 12:15:53.266077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.266731 (+   654us) service_pool.cc:224] Handling call
0709 12:15:54.932237 (+1665506us) spinlock_profiling.cc:243] Waited 61 ms on lock 0x7b540043c1d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:54.957723 (+ 25486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140245760}
I20260709 12:15:54.959021 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.960394 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.961099 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.964147 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.925089 29770 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:54.965551 29771 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:54.966140 29770 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.966360 29771 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.967629 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:54.903920 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 94675.5us
I20260709 12:15:54.968617 29770 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.958443 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:54.972806 23388 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:54.946234 29990 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.968982 23394 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:54.965173 29599 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:54.986368 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 71770.4us
I20260709 12:15:54.995803 23392 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:54.996222 29642 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:54.996840 29655 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:54.997789 29655 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.956893 29789 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.000953 29789 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:55.005972 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.006331 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.006955 23320 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:15:55.007664 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2813) took 1051 ms. Trace:
I20260709 12:15:55.008423 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.009022 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.010264 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.010946 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.011912 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:55.012423 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.012892 23315 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:54.969317 29770 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:54.904814 29989 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:54.914732 29697 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:54.901396 23462 rpcz_store.cc:276] 0709 12:15:52.283924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.408989 (+125065us) service_pool.cc:224] Handling call
0709 12:15:52.409369 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:52.473932 (+ 64563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:52.473962 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:53.576883 (+1102921us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:53.577227 (+   344us) log.cc:844] Serialized 26 byte log entry
0709 12:15:53.994951 (+417724us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:54.247629 (+252678us) log.cc:844] Serialized 14 byte log entry
0709 12:15:54.248894 (+  1265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:54.899828 (+650934us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b5400186fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:54.899952 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:54.900039 (+    87us) raft_consensus.cc:1685] finished
0709 12:15:54.900059 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:54.900462 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9767,"prepare.queue_time_us":306215,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":37,"raft.queue_time_us":949,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6220,"spinlock_wait_cycles":175681152,"thread_start_us":617356,"threads_started":2,"wal-append.queue_time_us":467423}
W20260709 12:15:55.006249 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:54.968647 29771 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.009281 23320 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:55.022902 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 2989) took 1425 ms. Trace:
I20260709 12:15:55.026491 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.017644 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.008162 23464 rpcz_store.cc:276] 0709 12:15:53.955776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.956024 (+   248us) service_pool.cc:224] Handling call
0709 12:15:53.987183 (+ 31159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:54.084094 (+ 96911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.007131 (+923037us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b540044af54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:55.007457 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145947904}
W20260709 12:15:55.032982 29998 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:15:55.015661 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.016253 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.015654 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.026108 29868 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:54.997071 29642 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.047900 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 2972) took 1509 ms. Trace:
I20260709 12:15:55.014739 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.032956 23393 rpcz_store.cc:276] 0709 12:15:53.597569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.597817 (+   248us) service_pool.cc:224] Handling call
0709 12:15:53.598101 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.815425 (+217324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.022374 (+1206949us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54001973d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:55.022740 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498455296}
I20260709 12:15:55.025175 29697 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:55.053208 23316 rpcz_store.cc:276] 0709 12:15:53.512777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.516681 (+  3904us) service_pool.cc:224] Handling call
0709 12:15:53.516956 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:53.796326 (+279370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.021425 (+1225099us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54002422d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:15:55.021789 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574875520}
I20260709 12:15:55.024227 29989 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.024549 29894 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:55.022239 29770 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.070051 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.045854 29655 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:51.664233 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:55.071157 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.025846 29779 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.085233 29779 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.025604 29997 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.085678 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.087508 29997 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.081462 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.088425 23314 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:55.089953 24008 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:15:55.090229 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.044065 29771 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.095705 30002 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.096529 30002 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.097342 29770 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.067119 29697 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:55.106174 23287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.614s (SENT) [suppressed 5 similar messages]
I20260709 12:15:55.108835 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.110153 29399 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.112666 29399 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.107995 29596 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.114276 29596 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.115145 29399 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.116688 29596 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:55.075881 29868 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:55.117904 29771 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.126582 30002 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.127286 29697 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.130815 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.133330 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.121534 23362 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:55.142407 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.135838 29868 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.147022 23464 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:55.146466 29771 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.103061 29894 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.151733 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 2976) took 2251 ms. Trace:
I20260709 12:15:55.152227 29642 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:55.152549 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 85619.7us
I20260709 12:15:55.152241 23470 rpcz_store.cc:276] 0709 12:15:52.899878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.901212 (+  1334us) service_pool.cc:224] Handling call
0709 12:15:53.273657 (+372445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.274271 (+   614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.150156 (+1875885us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54002265d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:55.151549 (+  1393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855686528}
I20260709 12:15:55.161343 29779 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:55.163086 29779 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.165864 29779 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.175019 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:55.175518 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.175915 23465 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:55.176579 29821 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:55.183444 30001 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.184276 30001 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.186676 30001 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:55.191617 29399 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.192417 29399 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.195325 29399 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.195708 29894 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.197788 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.198609 24007 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:55.199532 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:55.200392 24009 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:55.201263 29950 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.202245 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:15:55.201982 29950 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.204216 29950 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:55.204736 29950 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.176999 29821 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:55.205302 29821 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:55.205389 29950 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.207808 29950 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:55.209056 23396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2990) took 1610 ms (client timeout 1614 ms). Trace:
W20260709 12:15:55.209648 23396 rpcz_store.cc:269] 0709 12:15:53.598474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.598584 (+   110us) service_pool.cc:224] Handling call
0709 12:15:55.150374 (+1551790us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54002c7254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:55.208886 (+ 58512us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1121923840}
I20260709 12:15:55.180117 29997 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:55.225404 29931 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:55.225978 29931 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:55.227129 29931 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.234300 29642 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.235248 29642 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.257133 29929 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.258122 29929 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.260079 29655 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:55.260856 29655 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.271674 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.168887 29874 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:55.314347 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.317973 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.318116 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:55.324466 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 147027us
I20260709 12:15:55.328150 23318 raft_consensus.cc:2368] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:55.329489 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.356813 29674 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:55.357702 29674 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.360118 29674 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.361418 29929 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.362644 30001 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.363340 30001 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:55.365728 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.367174 30001 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:55.377718 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.380513 29642 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.384914 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.387952 29697 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.390139 29874 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:55.400274 23390 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 2993) took 1796 ms (client timeout 1884 ms). Trace:
W20260709 12:15:55.400887 23390 rpcz_store.cc:269] 0709 12:15:53.603608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.603750 (+   142us) service_pool.cc:224] Handling call
0709 12:15:55.399815 (+1796065us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54000a7b54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:15:55.400026 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968334720}
W20260709 12:15:55.457839 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.459517 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 356085us
I20260709 12:15:55.461910 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.473579 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:55.475308 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 142805us
W20260709 12:15:55.470561 23389 consensus_meta.cc:287] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.008s
W20260709 12:15:55.481760 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 292822us
I20260709 12:15:55.497278 29892 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:55.498173 29892 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.500675 29892 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.501387 23321 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:55.502261 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.502959 23469 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:55.503705 29913 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:55.505196 23389 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 12:15:55.505719 23389 pending_rounds.cc:85] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 9
I20260709 12:15:55.506176 23389 pending_rounds.cc:107] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:15:55.506471 23389 raft_consensus.cc:2889] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:55.507073 29868 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.507351 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 3009) took 1050 ms. Trace:
I20260709 12:15:55.507691 23389 rpcz_store.cc:276] 0709 12:15:54.457151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.457555 (+   404us) service_pool.cc:224] Handling call
0709 12:15:54.517938 (+ 60383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:55.507019 (+989081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.507303 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:15:55.508327 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:55.508914 29313 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.518121 23316 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:15:55.519609 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.520426 23468 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:15:55.526888 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.527546 23465 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:55.528479 29895 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.531643 29488 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.522040 23360 leader_election.cc:400] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:55.534368 23360 leader_election.cc:403] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:15:55.534899 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58698.2us
I20260709 12:15:55.535931 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.536918 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.537911 30009 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:55.542560 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.544517 23980 leader_election.cc:400] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:55.544898 23980 leader_election.cc:403] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:15:55.546053 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.547102 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.547513 29990 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:55.548051 29990 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.548219 23390 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:55.548509 29990 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:55.551594 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.552924 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.554550 30010 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:55.555068 30010 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:55.555496 30010 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:55.560771 29655 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.529307 29895 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.571278 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.571924 23465 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:15:55.568543 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.574636 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:55.576018 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.586323 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.587430 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.556951 29821 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.577162 29488 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:15:55.655019 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 83923.8us
I20260709 12:15:55.658329 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:55.640693 29978 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.212s	user 0.011s	sys 0.078s
I20260709 12:15:55.687032 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:55.647535 29488 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:55.687494 29789 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.686s	user 0.007s	sys 0.010s
I20260709 12:15:55.701910 24010 raft_consensus.cc:2393] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:55.706326 23313 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:55.713557 29895 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.714030 29692 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:55.716292 29895 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.719034 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.720996 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.722241 23387 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:55.723170 29987 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.723943 29987 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.726527 23391 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:15:55.726955 23391 pending_rounds.cc:85] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 13
I20260709 12:15:55.727213 23391 pending_rounds.cc:107] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:15:55.727456 23391 raft_consensus.cc:2889] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:55.668865 30008 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:55.700685 24016 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:55.704913 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:55.736760 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.733546 29482 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:55.706764 29488 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:15:55.751543 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.746209 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.768647 23394 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
W20260709 12:15:55.774063 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.775665 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:55.777493 23464 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:15:55.778146 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 284223us
I20260709 12:15:55.785738 23388 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:15:55.785560 29482 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.786764 29412 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:55.787411 29412 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:55.773116 29895 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.795535 29895 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.811033 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.777170 29580 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:55.785084 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:55.852057 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:55.856805 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 282759us
I20260709 12:15:55.856863 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.856961 23464 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:15:55.856060 23393 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:55.840543 29412 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:55.877825 30007 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:15:55.868917 29913 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:55.884702 29931 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:55.894477 29934 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.895368 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2226) took 1630 ms. Trace:
I20260709 12:15:55.896425 29482 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.895737 23312 rpcz_store.cc:276] 0709 12:15:54.265042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:54.266081 (+  1039us) service_pool.cc:224] Handling call
0709 12:15:54.266477 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:54.276631 (+ 10154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:54.276660 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:54.284117 (+  7457us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.130328 (+846211us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b18003b7700. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:15:55.479160 (+348832us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:15:55.693557 (+214397us) log.cc:844] Serialized 14 byte log entry
0709 12:15:55.700410 (+  6853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.710197 (+  9787us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:55.879273 (+169076us) raft_consensus.cc:1685] finished
0709 12:15:55.881815 (+  2542us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:55.895213 (+ 13398us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17383,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":11801,"spinlock_wait_cycles":7410560,"thread_start_us":500655,"threads_started":2,"wal-append.queue_time_us":1320343}
I20260709 12:15:55.902300 29990 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.871371 29821 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:55.906723 23393 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:55.908175 29642 consensus_queue.cc:1059] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:55.909147 29642 raft_consensus.cc:3055] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:55.909533 29642 raft_consensus.cc:740] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:15:55.910176 30013 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:15:55.910566 29642 consensus_queue.cc:260] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.911936 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1421) took 2291 ms. Trace:
I20260709 12:15:55.919868 23469 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:15:55.920456 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45522.8us
I20260709 12:15:55.921742 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.922443 23316 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:55.918872 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.876785 29987 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:55.929019 23388 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:15:55.931942 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:55.936878 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.913488 24012 rpcz_store.cc:276] 0709 12:15:53.620949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:53.621294 (+   345us) service_pool.cc:224] Handling call
0709 12:15:53.621645 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:53.621987 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:53.622020 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:55.269019 (+1646999us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b2c000900a8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:15:55.269740 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 12:15:55.580880 (+311140us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:15:55.582632 (+  1752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:55.603038 (+ 20406us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:15:55.909365 (+306327us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:15:55.911585 (+  2220us) raft_consensus.cc:1685] finished
0709 12:15:55.911609 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:15:55.911742 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":460,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9070,"spinlock_wait_cycles":32563712,"thread_start_us":302901,"threads_started":1,"wal-append.queue_time_us":474198}
W20260709 12:15:55.940804 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.943405 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.943861 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.944307 23392 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:15:55.945623 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:55.945693 29599 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:55.947018 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:55.948092 29812 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:55.948493 29812 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:55.948881 29812 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:55.948789 29764 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:55.953377 23313 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:55.951203 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:55.958868 23393 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:55.960181 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.913451 29913 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:55.961391 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.964483 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.965796 29894 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:55.971309 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.971966 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.990099 23465 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:15:55.972129 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.972880 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.973161 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.973606 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.973706 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:55.974375 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:55.981719 23314 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:15:55.982352 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:55.987670 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:55.992280 23464 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:15:55.991357 23468 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:15:55.994442 23462 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:15:55.981741 29642 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:55.994231 29313 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:55.989481 29894 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:55.997255 23470 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:15:55.989111 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.000077 29764 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:55.990646 30009 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:55.986899 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.004674 30015 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.003002 29764 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.005962 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:56.007767 30022 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:15:56.007740 29313 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.017863 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.008821 29987 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:56.010018 23392 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:56.013926 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:56.006933 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.016139 23469 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:15:56.003448 29894 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:56.016981 29721 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.017006 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.008886 30015 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.005234 29412 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:56.023608 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.011816 29764 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.025947 23390 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
W20260709 12:15:56.024607 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66641.5us
I20260709 12:15:56.034407 29801 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:56.034843 29801 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.035212 29801 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.024478 29987 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:56.040278 29987 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:56.030757 29868 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:56.040974 29868 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.041321 29868 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.045512 29721 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.046155 23321 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:56.033564 29412 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.047307 29412 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:56.030845 29599 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:15:56.047781 29599 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.047832 29721 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.048271 29599 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.006816 29692 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:15:56.050051 29844 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:56.050486 29844 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.050913 29844 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:56.061522 29962 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:56.060441 23214 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 15 to 16, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 16 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:56.062142 29962 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:56.062695 29962 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.065810 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.066956 29934 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:56.067574 29934 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.068034 29934 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.070968 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:56.088371 29313 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.088750 30015 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.063755 23214 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 9 to 10, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 10 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:56.089829 29789 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:56.091187 29913 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:56.088788 23216 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 344) took 53153 ms (client timeout 15000 ms). Trace:
W20260709 12:15:56.093323 23216 rpcz_store.cc:269] 0709 12:15:02.935902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:02.936146 (+   244us) service_pool.cc:224] Handling call
0709 12:15:03.133574 (+197428us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a65500b2 00007ff0af4c9759 00007ff0af461a76 00007ff0af432d58 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc
0709 12:15:56.088476 (+52954902us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1242420,"reported_tablets":48,"spinlock_wait_cycles":8188160}
I20260709 12:15:56.094432 29895 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:56.087585 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 81300.4us
I20260709 12:15:56.096432 23214 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 15, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
W20260709 12:15:56.099495 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 81711.9us
I20260709 12:15:56.102973 23214 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 8 to 9, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 9 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:56.105724 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.106472 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.106974 23214 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 10 to 11, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 11 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:56.109691 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.113039 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.115630 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.118393 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.121243 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.121999 24008 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:56.033763 29580 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
W20260709 12:15:56.097137 30010 consensus_meta.cc:287] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
I20260709 12:15:56.082708 30009 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:56.129171 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.129577 29990 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:56.130934 29817 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.132140 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.133677 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.133759 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.134006 29812 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.170287 23469 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:56.171511 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.171794 29812 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.135251 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.173035 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.173277 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.173386 29817 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.173982 24016 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:56.174690 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.139093 29894 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.139473 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.141319 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.141779 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.143076 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.175958 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.145164 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.146629 29488 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.147277 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:56.148509 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77626.5us
I20260709 12:15:56.149123 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:56.149521 29943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.697s	user 0.051s	sys 0.059s
I20260709 12:15:56.151074 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.151592 29987 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.152539 29868 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.194413 29764 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.194756 29868 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.194862 29894 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.152968 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.154861 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.157627 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.159344 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.168351 30009 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.162091 29412 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.199844 30009 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.201565 29412 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.201718 29990 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:56.101279 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 71090.6us
W20260709 12:15:56.202996 29943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.884s	user 0.051s	sys 0.060s
I20260709 12:15:56.204600 29913 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:56.128455 23360 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:56.163578 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 138170us
I20260709 12:15:56.142910 29599 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.174624 24012 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:56.196650 29764 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.190582 29817 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:56.191046 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.192690 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.192884 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.198921 29844 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.159375 29934 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.202361 29990 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.211107 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.212296 23462 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:56.214730 24011 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:15:56.220322 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.220929 23316 raft_consensus.cc:2410] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:15:56.223513 29599 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.168237 30010 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.225019 23389 raft_consensus.cc:2368] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
I20260709 12:15:56.197152 29801 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.226151 30010 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.194778 29987 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.227324 29962 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.228061 29801 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.228806 24008 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:56.228932 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.231041 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.231297 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.235708 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.237551 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.238624 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.238893 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.242317 23464 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:56.244197 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:56.246865 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 145829us
I20260709 12:15:56.244846 23470 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:56.247027 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.248531 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.248932 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.249514 23466 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:56.233136 29962 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.256491 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.257777 24009 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:15:56.258651 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.259662 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.259814 29943 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.260574 29943 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.262504 29934 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.256477 29913 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:56.263398 29913 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.263784 29913 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:56.265662 30025 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:15:56.266136 30025 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:56.266731 30025 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:56.267855 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.263227 29943 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.253808 30026 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:56.269972 30026 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:56.270517 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.270943 30026 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.274425 23287 leader_election.cc:400] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
I20260709 12:15:56.274709 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.275285 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.275734 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.278391 30010 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.279083 30010 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.274729 23287 leader_election.cc:403] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:56.282155 29764 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:15:56.282348 29987 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:56.282454 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:56.283067 29987 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.283510 29987 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:56.238229 29894 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.293406 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.296603 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.297204 23314 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:56.286098 29894 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.286972 30010 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.299192 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.301316 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.305732 23321 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:56.305975 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.306581 23467 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.308266 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.309401 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.312064 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.314178 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.296465 29990 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:56.325615 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.329027 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.308139 29943 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.310695 29934 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:15:56.338691 29949 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:56.339196 29949 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:56.339581 29949 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.340533 29764 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
I20260709 12:15:56.341940 29488 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.346995 29894 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.325351 30027 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.348304 30027 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.350479 30027 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.351948 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.352643 23389 raft_consensus.cc:2368] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:56.353315 29672 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:56.353730 29672 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:56.354081 29672 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.357702 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.307880 30009 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.359691 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.360010 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.360929 30009 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.360896 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.361884 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.363001 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:56.363839 29899 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:56.364178 29899 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.364507 29899 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.365542 29412 consensus_queue.cc:1059] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:56.356389 29868 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:56.366387 29789 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:56.367075 29789 consensus_queue.cc:1243] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:56.368146 29789 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:56.368510 29789 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.368849 29789 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.369608 29599 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:56.370020 29844 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.370370 29915 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:56.370807 29915 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.371160 29915 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.370024 29599 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:56.371837 29482 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:56.372190 29599 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:56.372254 29482 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:56.373466 29482 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.368784 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.357609 29990 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:56.374585 23315 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:56.375001 29990 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:56.375425 29990 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.377543 29943 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.380529 29943 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.384898 23314 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:56.385720 23321 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:56.385895 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.391116 23312 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
W20260709 12:15:56.391525 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.392571 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:56.391440 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65808.8us
I20260709 12:15:56.395555 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.395666 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.396126 23315 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.398130 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.398815 23390 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:15:56.399348 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.402215 29987 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.403873 29913 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.405948 29913 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.414940 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.415704 23390 raft_consensus.cc:2368] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:56.417649 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:56.419390 29764 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:56.397186 29943 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:56.426812 23467 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:15:56.427521 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.427709 23392 raft_consensus.cc:2410] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:15:56.408231 29987 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.432369 23388 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:56.432271 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.445276 23394 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:56.430651 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.433686 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.438807 29915 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.439554 29943 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.440487 29949 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.449087 23389 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:15:56.450502 29943 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:56.451534 29672 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.456138 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:56.457103 23391 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:15:56.458604 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.459389 23391 raft_consensus.cc:2393] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:56.460249 29892 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:56.460813 29892 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.461287 29892 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:56.464234 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:56.465046 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.468492 23315 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:56.472704 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.473538 23394 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:56.474196 29934 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:56.476723 29789 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.479885 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.493865 23394 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.495635 29789 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.480564 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.480278 29990 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.489831 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.489964 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.484685 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.489351 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.497885 23388 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:56.501700 23360 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.506384 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.507021 23470 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.507709 24014 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:56.508903 29721 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:56.509665 23389 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:56.512588 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:56.507287 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:56.513504 23314 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:56.505510 23360 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.512665 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:56.521551 29721 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:56.522540 29672 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.522689 29721 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.526476 29987 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:56.527025 29949 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:56.504352 29599 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.531075 29599 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:56.532409 29915 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.527099 29987 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:56.547312 23360 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.555342 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.559165 30010 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:56.559737 30010 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:56.560613 30010 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.562024 29990 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:56.562883 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.556842 23360 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.564427 23470 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:56.567037 30009 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:56.567562 30009 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:56.568147 29987 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.568455 30009 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.578298 23389 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:15:56.580984 23314 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:15:56.586160 23392 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 21. Current term is 22. Ops: []
W20260709 12:15:56.586540 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 79345.2us
W20260709 12:15:56.579664 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 125185us
I20260709 12:15:56.549937 29913 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:56.610855 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.585487 29892 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.627875 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.588152 23360 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:56.634794 29997 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:56.635387 29997 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:56.643939 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 49245.7us
I20260709 12:15:56.648547 23394 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:15:56.642951 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 310231us
I20260709 12:15:56.649909 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:56.653077 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:56.662647 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.663762 23389 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.611874 29990 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:15:56.676836 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:56.682346 23388 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:56.678021 29596 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:56.656728 29949 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:56.685581 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:56.687453 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:56.685855 29672 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:56.698843 23392 raft_consensus.cc:1240] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:56.701655 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.706292 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.706677 29892 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:56.707329 23391 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:56.710255 29844 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:56.710134 29997 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.710829 29844 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:56.711743 29844 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.723213 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.709090 29482 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:56.723213 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.726461 23468 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:56.750671 23468 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 21. Current term is 22. Ops: []
I20260709 12:15:56.750666 23470 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:56.789834 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:56.776614 30025 consensus_meta.cc:287] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.509s	user 0.014s	sys 0.000s
I20260709 12:15:56.792075 23389 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:56.792140 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:15:56.792825 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73317.3us
I20260709 12:15:56.793053 23468 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:15:56.797335 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46019.3us
I20260709 12:15:56.722632 29899 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:56.806463 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.808573 23470 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:56.811957 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:56.780354 29949 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:15:56.814181 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 62704.4us
W20260709 12:15:56.862401 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50925.6us
I20260709 12:15:56.862452 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:56.864311 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58441.8us
I20260709 12:15:56.865443 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.867596 23467 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:56.868901 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.870036 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.870261 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.871704 23470 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
W20260709 12:15:56.861838 30045 log.cc:927] Time spent T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:15:56.873634 23462 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:56.963793 24008 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:56.964450 24010 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:56.938576 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:56.936235 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 65605.1us
I20260709 12:15:56.939433 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.938918 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.875417 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.954648 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.956446 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.967293 24015 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:56.960342 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:56.961321 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:56.962782 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:56.963111 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.800894 30025 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:56.966847 23463 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:15:56.995811 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:56.997013 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:56.936154 29599 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:15:57.005637 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.002681 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.011394 23462 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:15:57.011878 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.012559 23464 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:15:57.013949 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:57.016319 23390 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:56.872684 29949 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.020676 23362 leader_election.cc:400] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:57.025278 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:57.015868 29943 consensus_meta.cc:287] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:15:57.026582 29943 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.027760 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.028067 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:57.029871 23470 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:15:57.031008 23469 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 17. Current term is 18. Ops: []
W20260709 12:15:57.036350 30050 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:15:57.038801 29899 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:57.020967 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 158612us
I20260709 12:15:56.950527 29764 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 12:15:57.075356 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:57.073467 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 105882us
I20260709 12:15:57.073150 29894 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:15:57.104959 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 79758.9us
I20260709 12:15:57.106069 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.073496 29844 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:57.131332 23468 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
W20260709 12:15:57.131407 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65397.5us
I20260709 12:15:57.095597 29899 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:57.151805 23466 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:57.130146 30022 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:57.156433 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:57.152400 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.170739 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.174212 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:57.177996 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 44544.2us
I20260709 12:15:57.144761 29844 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.169555 29987 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:57.177124 29764 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:15:57.189417 30053 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:15:57.192359 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.193765 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.194813 29599 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:57.166502 30026 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:15:57.194522 29913 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:15:57.212232 30025 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.183614 30059 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.215451 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.216756 30059 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.217351 29913 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:15:57.217808 29913 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.218186 29913 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:57.227056 30026 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:57.228334 29943 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.227957 30026 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.228540 29949 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.230433 30026 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.167289 29899 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.227798 23436 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.233058 29899 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.234126 29789 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:15:57.235100 29844 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.182432 29997 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.240473 29913 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.243362 23321 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:57.246153 23319 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:15:57.187889 29987 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:57.189195 29894 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:15:57.247764 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.265658 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.266116 23313 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:57.266341 23470 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:15:57.258653 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.264165 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.264784 24015 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
W20260709 12:15:57.264230 30046 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.399s	user 0.000s	sys 0.004s
W20260709 12:15:57.276507 29482 consensus_meta.cc:287] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.004s
I20260709 12:15:57.278211 29596 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:57.279284 29580 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:15:57.279776 29580 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.280994 29580 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.282976 29580 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:15:57.284416 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.252306 30059 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:57.298208 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110126us
I20260709 12:15:57.268637 29987 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:57.268512 29894 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:57.302562 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 128449us
I20260709 12:15:57.297209 29672 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.305007 24009 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:57.305691 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:57.306586 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.372617 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:57.372999 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 66482.8us
I20260709 12:15:57.373993 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.375725 23436 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:15:57.301668 30009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.000s	sys 0.009s
I20260709 12:15:57.307571 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.302544 29894 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:15:57.376472 30009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.000s	sys 0.009s
I20260709 12:15:57.307215 29412 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
W20260709 12:15:57.377028 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69538.5us
I20260709 12:15:57.322508 23313 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:57.318974 30035 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 12:15:57.342525 30058 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 12:15:57.378139 29789 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:57.378310 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.378624 29789 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.379048 29789 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:57.379659 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.380517 30059 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:57.380877 30059 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:57.381225 30059 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:57.345741 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.381601 29915 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:57.382066 23314 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:57.382354 29915 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:57.382720 29915 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.383037 29915 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:57.383886 30066 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:57.384331 30066 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:57.384677 30066 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:57.386848 30044 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:15:57.387245 30044 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.388116 30044 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.389890 23314 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:57.364078 29997 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:15:57.364398 30063 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:57.367832 23387 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:15:57.369777 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.370077 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.371668 23389 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:57.389709 29899 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:57.354462 29913 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:57.390326 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.390755 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.390877 29892 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.391359 29992 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:57.389302 30044 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:57.393162 29894 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.402225 29599 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:57.426728 29599 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.427850 29599 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.402931 30061 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:57.430159 30061 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:57.430641 30061 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:57.433461 29599 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:57.444247 30061 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.404415 29992 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.446676 29992 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.404666 29899 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:57.448391 29899 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.448752 29899 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:57.400076 29915 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.452437 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:15:57.453588 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.454653 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:57.404987 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.409795 23215 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 6 to 12, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 12 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:57.458344 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53446.3us
I20260709 12:15:57.458395 23313 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:15:57.459363 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.459226 23215 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 16 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:57.460152 30071 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:15:57.411916 23214 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 345) took 30649 ms (client timeout 15000 ms). Trace:
I20260709 12:15:57.463538 29913 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:15:57.423363 30063 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:57.465274 30063 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:57.465689 30063 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:57.467936 29899 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.413456 30066 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.424228 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.422744 29789 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.476097 30063 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.408826 29894 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:57.460651 23215 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 17, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.478674 23215 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:15:57.461902 23214 rpcz_store.cc:269] 0709 12:15:26.771883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:26.772175 (+   292us) service_pool.cc:224] Handling call
0709 12:15:27.723496 (+951321us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a65500b2 00007ff0af4c9759 00007ff0af461a76 00007ff0af432d58 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc
0709 12:15:57.411806 (+29688310us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:56.377249 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:56.377982 (+   733us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:56.378038 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:15:56.378072 (+    34us) tablet.cc:662] Decoding operations
0709 12:15:56.378829 (+   757us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:56.378993 (+   164us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:56.379022 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:15:56.379875 (+   853us) tablet.cc:701] Row locks acquired
0709 12:15:56.379937 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:15:56.380188 (+   251us) write_op.cc:270] Start()
0709 12:15:56.380370 (+   182us) write_op.cc:276] Timestamp: P: 1783599356380101 usec, L: 0
0709 12:15:56.380414 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:15:56.426135 (+ 45721us) log.cc:844] Serialized 2099 byte log entry
0709 12:15:56.797178 (+371043us) op_driver.cc:464] REPLICATION: finished
0709 12:15:57.114606 (+317428us) write_op.cc:301] APPLY: starting
0709 12:15:57.133312 (+ 18706us) tablet.cc:1367] starting BulkCheckPresence
0709 12:15:57.192023 (+ 58711us) tablet.cc:1370] finished BulkCheckPresence
0709 12:15:57.192133 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:15:57.195358 (+  3225us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:15:57.195652 (+   294us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 12:15:57.195737 (+    85us) write_op.cc:312] APPLY: finished
0709 12:15:57.196075 (+   338us) log.cc:844] Serialized 57 byte log entry
0709 12:15:57.197185 (+  1110us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:57.197308 (+   123us) write_op.cc:454] Released schema lock
0709 12:15:57.197366 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:15:57.197825 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:15:57.197851 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1127100,"reported_tablets":48,"spinlock_wait_cycles":61999744,"child_traces":[["op",{"apply.queue_time_us":274611,"apply.run_cpu_time_us":10165,"apply.run_wall_time_us":118201,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":210085,"prepare.run_cpu_time_us":29171,"prepare.run_wall_time_us":231594,"raft.queue_time_us":211100,"raft.run_cpu_time_us":50657,"raft.run_wall_time_us":141688,"replication_time_us":416577,"thread_start_us":485902,"threads_started":4,"wal-append.queue_time_us":98359}]]}
I20260709 12:15:57.473145 30066 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.425518 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.400929 30059 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.484141 23215 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 18 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.486421 30059 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.403573 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:57.491060 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 87568.8us
I20260709 12:15:57.424801 29987 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.495581 29987 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.496321 29913 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.491528 30075 consensus_queue.cc:1059] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:15:57.497242 30075 consensus_queue.cc:1243] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:57.497586 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.497421 29913 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.497920 30075 consensus_queue.cc:1059] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:57.498445 30076 raft_consensus.cc:3055] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:57.492434 30071 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.493199 30059 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:57.493695 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.501864 30059 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:57.502547 30059 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:57.501760 30071 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.500470 30076 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.503521 30059 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.505092 30076 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.506628 30076 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:15:57.423553 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 117925us
I20260709 12:15:57.506681 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.508112 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.508297 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.509011 23313 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:57.509554 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.510018 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.509161 30063 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.511327 30063 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.491390 23215 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 19, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 19 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.511579 29894 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:57.512910 29997 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:57.513311 29997 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:57.513031 29894 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:57.514153 29894 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:57.514694 29894 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:57.514196 29997 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.515604 29997 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:57.516120 29894 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.517732 29894 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:57.518254 30079 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:15:57.519290 30079 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.485351 29899 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.536118 23215 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.532922 23360 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:15:57.546813 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.549818 23360 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:57.499752 30075 raft_consensus.cc:3055] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:57.568070 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.579324 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:57.567526 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 267211us
I20260709 12:15:57.572364 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.580595 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.577381 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.582278 23317 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:57.585377 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.586040 23394 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:57.585089 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.586844 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.592167 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.577057 29997 raft_consensus.cc:2804] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:15:57.594108 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.596942 23215 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 14, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 14 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:57.598693 29789 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.598502 23215 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.599273 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.600438 23215 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 13, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:57.595600 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.576689 29987 raft_consensus.cc:2804] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:15:57.602438 29987 raft_consensus.cc:697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:57.603374 29987 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.578300 30075 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.610213 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.585009 30027 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:57.630959 23466 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:57.553191 30079 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:15:57.595103 30015 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:57.583143 30071 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:15:57.635946 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.637547 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.637451 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.637759 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.579429 30009 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:15:57.643018 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.646329 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.643746 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:15:57.644694 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.631023 23215 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 13, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 13 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:57.626478 29580 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:57.648305 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:57.649669 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.650135 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.650363 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.651047 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:57.651921 23315 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:57.635996 29915 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.701296 24009 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:57.703835 24008 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:15:57.704232 23394 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:15:57.632725 29997 raft_consensus.cc:697] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:57.707931 24008 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:57.653523 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.654016 23317 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:57.714421 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.714499 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.715579 23319 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:57.716149 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.716641 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.717233 23317 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:15:57.654592 24014 raft_consensus.cc:2368] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:15:57.703518 29672 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:15:57.719128 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.720244 29672 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:57.720578 23315 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:15:57.654664 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.608868 30067 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.655398 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.655777 29789 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:57.647426 30022 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:15:57.656435 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46319.4us
I20260709 12:15:57.743285 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.656982 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:57.657696 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:57.658406 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.658622 29844 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:57.746191 29844 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:57.746088 29789 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.746726 29844 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.747769 29844 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.748090 29997 consensus_queue.cc:237] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.658443 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:57.750044 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 91680.5us
I20260709 12:15:57.752243 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.661015 29992 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:57.662200 23215 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.663074 30061 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:57.664939 30075 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.754199 30075 raft_consensus.cc:3060] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:57.755025 23362 leader_election.cc:400] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:57.754415 23215 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 14, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 14 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.756577 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:57.668910 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:57.667290 24007 raft_consensus.cc:1240] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:57.757604 30022 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.671027 24012 raft_consensus.cc:1240] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:57.758322 30022 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:15:57.758685 30022 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:57.673444 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:57.675165 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.759857 23388 raft_consensus.cc:2393] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:57.676640 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:57.690415 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:57.692215 30071 consensus_queue.cc:1243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:15:57.691738 30059 consensus_queue.cc:1059] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:15:57.653043 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 359066us
I20260709 12:15:57.706349 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.736588 23317 raft_consensus.cc:1240] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:15:57.738168 23393 raft_consensus.cc:1240] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:57.738502 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:15:57.765842 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59578.1us
I20260709 12:15:57.738760 24010 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:57.700399 30079 consensus_queue.cc:1243] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:15:57.742452 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.756011 23215 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 17 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.670904 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:57.769663 23390 raft_consensus.cc:2393] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:57.757509 23319 raft_consensus.cc:1240] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 13. Current term is 14. Ops: []
I20260709 12:15:57.759799 24009 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:15:57.696314 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.753492 23313 raft_consensus.cc:2393] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:15:57.770228 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.720598 29672 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.764931 29580 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:15:57.759112 30022 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:15:57.788197 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 136575us
I20260709 12:15:57.749326 29844 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:57.794279 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98048.7us
I20260709 12:15:57.795586 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.791759 23215 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.797838 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.777480 30071 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:57.810477 23215 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.815840 30026 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:57.771641 29412 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:15:57.771497 29482 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:15:57.833894 30071 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.817109 29599 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:57.844188 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46443.6us
I20260709 12:15:57.845034 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.826848 29987 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:57.798595 30065 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.805076 30066 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:15:57.854216 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58716.8us
I20260709 12:15:57.816484 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.855842 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.807832 30079 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:15:57.842501 30071 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.859949 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:57.862979 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46575.4us
I20260709 12:15:57.865698 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:57.842295 29599 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.871606 29599 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.858703 29672 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.874150 30069 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:57.874749 30069 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.875754 30069 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.836447 30035 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:15:57.877538 30070 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:57.878803 29596 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:57.879185 29596 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:57.879868 30079 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:57.880074 29596 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:57.859031 30063 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:57.865530 23215 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.890903 30074 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:57.891433 30074 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:57.889231 23215 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 8 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.867708 29913 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:57.904554 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 113354us
I20260709 12:15:57.895051 29987 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:57.901726 23215 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 13, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.889423 29789 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.897943 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:57.910408 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:57.897498 29599 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:15:57.913390 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:57.914260 23390 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:15:57.912168 23215 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:57.912588 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 121664us
I20260709 12:15:57.922639 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.920269 23215 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:15:57.929905 23434 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:57.928185 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.935091 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:57.936137 23215 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.938709 23215 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 12 to 17, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:15:57.900035 29990 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:15:57.946120 23215 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:15:57.956245 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 100380us
I20260709 12:15:57.960635 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:57.963270 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.963964 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:57.934650 29789 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:57.935366 30073 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:15:57.967257 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 107375us
I20260709 12:15:57.971203 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:57.950642 30076 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:57.983546 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.983978 23466 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:57.980834 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.981855 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:57.983806 23463 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:15:57.973284 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:57.951936 29599 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:15:57.994385 23461 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
I20260709 12:15:57.998811 23469 raft_consensus.cc:1240] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:57.999380 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.001178 23468 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:15:58.002086 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.002653 23461 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:57.992305 29580 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:57.966988 30079 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:57.927116 30074 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:58.030941 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102857us
I20260709 12:15:57.997249 29599 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.993006 30065 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:15:58.054858 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 71396.8us
I20260709 12:15:58.059023 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.023877 30059 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.025486 30015 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.056665 29599 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:58.068996 29482 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 12:15:58.075047 30091 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:15:58.068827 30102 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:15:58.076144 29844 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:15:58.080159 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.082708 30026 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:58.069128 29580 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:15:58.129944 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:58.178733 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.195736 29899 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:15:58.212199 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:15:58.393855 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:15:58.225090 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:58.228389 30083 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:15:58.238011 30067 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:58.400035 30067 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:58.401240 30067 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.402779 30067 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.380611 30102 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:58.403504 30102 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:15:58.244196 30074 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:58.213339 30010 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:58.404417 30010 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.272239 23389 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:58.405586 29899 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:58.406651 30010 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.272742 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.273407 29997 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:58.407536 29580 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.408082 29997 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:58.274055 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 280434us
I20260709 12:15:58.410266 29997 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:15:58.306545 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.310833 29672 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:58.413057 29844 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.406587 29899 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.414623 29899 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:15:58.415294 29844 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:58.380920 29913 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:58.416492 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:58.331082 29992 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.003s
W20260709 12:15:58.331961 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.340360 30111 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.421875 30111 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.348227 23461 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
W20260709 12:15:58.362068 30022 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.003s
I20260709 12:15:58.424929 30022 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.427062 30022 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.367002 30068 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:58.035530 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 62582.5us
W20260709 12:15:58.378142 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 319205us
I20260709 12:15:58.378302 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:58.379488 23387 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:15:58.390677 23392 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:58.391690 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.396133 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.396674 30075 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:58.428514 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.433202 23464 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:58.437068 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.440883 30111 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.438113 30068 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.401243 29599 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:58.535183 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 98184.4us
I20260709 12:15:58.535845 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:58.538071 30059 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:58.541186 29990 consensus_queue.cc:1059] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:15:58.541612 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.542058 29990 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:58.542425 29990 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:58.543303 29990 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.544574 29990 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:58.546944 29672 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:15:58.549024 30059 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:58.537703 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 109272us
W20260709 12:15:58.537024 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 145409us
I20260709 12:15:58.540262 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.561372 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.556814 30061 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:58.564864 23463 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:15:58.536538 23389 raft_consensus.cc:1240] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:58.558808 23392 raft_consensus.cc:1240] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:58.559393 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.560336 23465 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
I20260709 12:15:58.538825 30075 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:58.562649 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.562901 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.536998 30073 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:58.569331 29949 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.569039 23464 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:58.570120 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:58.536430 29913 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:15:58.578883 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.581113 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:58.582641 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:58.583135 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.584264 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:58.591830 23467 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:58.592974 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:58.571161 29949 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.594607 24012 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:58.565503 29894 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.569026 29899 raft_consensus.cc:3060] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:58.575707 30065 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:15:58.594631 29599 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:58.565166 30061 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.571214 29997 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:58.567799 30103 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:58.571269 30074 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:15:58.615572 23461 raft_consensus.cc:2468] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:15:58.625730 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.627381 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:58.628328 23461 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:58.573158 30010 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:15:58.631920 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.638717 23461 raft_consensus.cc:2393] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 21: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:58.596339 30097 consensus_queue.cc:1059] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:15:58.646196 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.623862 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.604820 30059 consensus_queue.cc:1059] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:15:58.650285 23314 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:58.646471 30015 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.625182 30102 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:58.655678 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 53311.9us
I20260709 12:15:58.656915 23320 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
W20260709 12:15:58.660147 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 97564.9us
I20260709 12:15:58.661342 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:58.660702 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.660401 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.639564 29412 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:58.666075 23314 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:58.663722 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:58.669333 23461 raft_consensus.cc:2393] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:58.667812 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.668377 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:58.669962 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.670073 23392 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:58.670135 23390 raft_consensus.cc:2376] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20. Re-sending same reply.
I20260709 12:15:58.671465 23467 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:15:58.671715 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.671864 23467 pending_rounds.cc:85] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 10
I20260709 12:15:58.672273 23467 pending_rounds.cc:107] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:15:58.672601 23467 raft_consensus.cc:2889] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:58.674552 23389 raft_consensus.cc:1240] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 22. Current term is 23. Ops: []
I20260709 12:15:58.678038 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:15:58.680325 23390 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:58.681759 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.658769 30009 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:15:58.683657 23388 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 12.
I20260709 12:15:58.683905 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.690090 23320 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:15:58.690260 23467 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:15:58.690861 29892 consensus_queue.cc:1059] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:58.691355 23461 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:58.693638 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.694234 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:58.700274 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.659081 30063 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:58.667411 30097 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:58.705024 23461 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:58.668008 30092 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.672679 29412 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:58.681809 30010 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.685321 30026 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:58.711647 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:58.682471 30095 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:15:58.710892 30063 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.698048 30009 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:58.716363 29949 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.697832 30074 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.657821 30015 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.715626 29892 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:58.731444 30102 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:15:58.731992 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 69944us
I20260709 12:15:58.751950 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.753177 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.706382 30097 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.717631 29844 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.724049 30095 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:58.728951 30114 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:15:58.765275 23393 raft_consensus.cc:2368] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:58.681630 30061 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:58.724977 30113 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.186s	user 0.002s	sys 0.000s
I20260709 12:15:58.755399 23436 leader_election.cc:400] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:15:58.771471 23436 leader_election.cc:403] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:58.720680 30092 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.743971 29892 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.754971 29789 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:15:58.743741 30119 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:58.724768 30010 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.780598 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:58.765574 30095 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:58.797536 30097 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.798823 30097 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:58.762795 30116 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:15:58.803505 29599 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:15:58.776165 30103 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.808308 30103 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.811286 30103 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.811807 30070 raft_consensus.cc:3055] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:15:58.812227 30070 raft_consensus.cc:740] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.812664 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:58.812377 30009 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.813524 24013 raft_consensus.cc:2410] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:15:58.813006 30070 consensus_queue.cc:260] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.814280 30009 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:58.762440 30068 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.764273 29949 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:15:58.816113 29949 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:58.816509 29949 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:58.811314 29892 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.818481 29892 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:15:58.789845 30112 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:15:58.775494 29412 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.791512 30063 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.793711 30015 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.764717 29844 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.826815 29949 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.827661 29844 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.828002 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:58.830248 29949 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:58.832134 30061 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:58.835098 30119 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.836287 30119 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.839224 30092 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:58.841507 30010 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.841506 30070 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:58.850338 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:15:58.850695 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 143592us
I20260709 12:15:58.851707 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:58.853865 30067 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:15:58.854550 30067 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:58.855504 30067 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.857859 23461 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:58.865742 30105 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:58.866248 29596 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:15:58.866302 30105 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.871037 30105 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.872836 29412 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:58.874532 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:58.876142 29672 consensus_queue.cc:1059] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:15:58.876832 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 121540us
I20260709 12:15:58.857272 30095 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.880496 23464 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:15:58.880991 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.853749 30059 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:58.906773 23467 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:58.908661 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:58.890894 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:58.891193 30065 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:15:58.906905 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:15:58.908236 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 224387us
I20260709 12:15:58.912714 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:58.914858 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.916592 24010 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:15:58.918455 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:58.893990 30015 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:58.918972 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.919037 30015 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.919781 23319 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:58.921406 30015 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:58.894604 30063 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:58.895911 29943 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:15:58.896785 30022 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:58.901753 30102 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:15:58.902698 29672 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:15:58.902922 30092 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:15:58.929396 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:58.903677 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:58.931900 30097 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:15:58.933928 30063 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:15:58.934368 30063 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:58.908564 23313 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:15:58.910044 23461 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:15:58.935314 30063 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.886188 23436 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:58.936663 30063 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:15:58.937341 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:58.937278 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 59580.7us
I20260709 12:15:58.938154 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:58.926595 30010 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:15:58.938938 30065 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:15:58.941922 23395 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 14. Current term is 15. Ops: []
I20260709 12:15:58.892678 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:58.943118 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50548.8us
I20260709 12:15:58.943564 30065 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:15:58.944435 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.945150 30095 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.945701 30022 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:58.946461 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:58.947064 23393 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:58.947602 23470 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:58.949748 30074 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:15:58.950649 30074 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:15:58.951141 30074 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:58.952008 30074 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.930583 23287 leader_election.cc:400] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
I20260709 12:15:58.953867 23287 leader_election.cc:403] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:15:58.925654 30059 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:58.947616 29899 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:58.954941 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:58.955890 30059 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:58.949851 30027 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:15:58.956435 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.956789 30027 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:15:58.957324 23320 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 13: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:58.930624 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.958182 30027 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.957741 30121 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:15:58.956463 30059 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:15:58.959616 30027 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:58.959661 30121 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:58.960886 30121 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:58.930641 30092 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:58.965869 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.966943 23314 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:15:58.967866 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.965619 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.964335 23387 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:15:58.967955 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:58.969316 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:58.969874 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:58.967355 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.971566 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.972669 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:15:58.973294 23318 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:15:58.973809 23318 pending_rounds.cc:85] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:15:58.974092 23318 pending_rounds.cc:107] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:15:58.974381 23318 raft_consensus.cc:2889] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:58.965073 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.922547 29997 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:15:58.985314 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:58.994314 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:58.995414 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.996789 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.999411 23394 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:58.999814 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.000679 23315 raft_consensus.cc:2410] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:15:59.002734 23388 raft_consensus.cc:2410] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:15:59.004048 23395 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:58.984787 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:58.930624 29987 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:15:59.008653 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43663us
I20260709 12:15:59.015534 30097 consensus_queue.cc:1059] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:15:59.015836 30022 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:15:59.016634 30022 consensus_queue.cc:1243] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:15:59.017038 30035 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:59.017329 30124 raft_consensus.cc:3055] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:15:58.994700 30015 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:15:58.980945 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77368.8us
I20260709 12:15:59.005851 29844 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:15:59.018599 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.017765 30124 raft_consensus.cc:740] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:59.000013 30066 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:15:59.008019 30068 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.979869 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:59.017490 30035 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.023269 23982 leader_election.cc:400] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17.
I20260709 12:15:58.993847 29899 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:15:58.999316 30061 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
W20260709 12:15:59.029135 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49364.1us
I20260709 12:15:59.010310 23466 raft_consensus.cc:2468] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:15:58.980260 23393 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:59.032027 23312 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:15:59.035099 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:59.042595 30124 consensus_queue.cc:260] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:59.047086 30125 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
W20260709 12:15:59.054495 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.054648 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.055747 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.019198 30095 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:15:59.066020 23461 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:15:59.022176 29844 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:59.017663 29987 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:15:59.060986 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:59.079162 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.060971 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.028684 30035 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:59.063339 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:15:59.073822 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77743.6us
I20260709 12:15:59.035801 30009 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:59.038177 30010 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:59.077994 24010 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:15:59.043293 29943 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:15:59.089046 23321 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:59.091511 24009 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:15:59.090979 23462 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:59.099015 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.100704 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:59.063102 29672 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.104182 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.119457 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.113703 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:15:59.120677 23394 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:15:59.103807 29992 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:59.114445 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.037180 30075 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:15:59.028538 30068 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.079151 29844 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:15:59.124979 23387 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
W20260709 12:15:59.144076 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.120095 23388 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
W20260709 12:15:59.144647 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.151727 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.154868 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.155258 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.161744 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.043778 29789 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:15:59.170269 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.028657 30092 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.149933 30111 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:15:59.177161 24009 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:59.143832 29672 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.123214 30121 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:15:59.187414 30116 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:59.118008 30105 consensus_queue.cc:1059] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:15:59.185061 23394 raft_consensus.cc:2468] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:59.117511 30022 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:15:59.101708 29943 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:15:59.188278 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.195456 30105 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:15:59.195729 29943 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:15:59.196012 30105 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:59.188655 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:59.197857 30061 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:59.199142 30116 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:59.200664 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:59.201926 30116 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:59.196897 30105 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.204387 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.200644 30061 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:15:59.207095 30103 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:15:59.208323 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:59.209322 30103 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:15:59.209895 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 119790us
W20260709 12:15:59.193300 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 157213us
I20260709 12:15:59.210357 23434 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:15:59.210918 29943 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.217418 30103 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:15:59.218497 23389 raft_consensus.cc:1240] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:15:59.218634 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.219461 29949 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:59.220258 29992 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:59.218966 29943 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.222548 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.223532 29599 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:15:59.225075 29599 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:15:59.225512 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.226769 29599 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:15:59.228094 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.229048 29949 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.229667 29949 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.232668 29949 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:59.235057 30142 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:59.235641 30142 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:59.236892 29987 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.236918 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.237833 30142 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.238611 30148 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:15:59.250391 24011 raft_consensus.cc:1240] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: []
I20260709 12:15:59.251943 24007 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:59.237905 29987 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.261220 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.262938 29482 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:59.263469 29482 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.266840 29482 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:15:59.265830 29987 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.271071 29599 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:59.205111 30105 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:15:59.280778 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.297734 23387 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 20: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:15:59.299139 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 177206us
I20260709 12:15:59.300179 23395 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:15:59.302109 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.302980 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.306490 23316 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:59.164906 30059 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19.
W20260709 12:15:59.299487 30126 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.180s	user 0.003s	sys 0.000s
W20260709 12:15:59.315390 30136 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:15:59.307312 29949 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:15:59.327878 30061 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:15:59.297813 29899 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:15:59.373656 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 273247us
I20260709 12:15:59.301002 30116 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:15:59.357447 29987 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:59.384972 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.417943 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.418303 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:59.423529 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 201038us
W20260709 12:15:59.425480 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 100736us
W20260709 12:15:59.431586 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 142310us
I20260709 12:15:59.438105 23314 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:15:59.481375 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:59.419768 29894 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:15:59.413573 29599 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:59.406616 30147 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.392675 29987 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:59.527626 30063 consensus_meta.cc:287] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.003s
I20260709 12:15:59.565910 23395 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:59.525270 30147 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.600306 30153 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:15:59.600976 30137 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:15:59.601332 30158 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.606897 30164 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:59.609699 30150 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:15:59.572536 30059 consensus_queue.cc:1059] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:15:59.593122 30068 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.589577 30163 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.588960 29672 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:15:59.659102 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 227965us
W20260709 12:15:59.668744 24013 consensus_meta.cc:287] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.001s
I20260709 12:15:59.675005 23467 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:15:59.618804 29482 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.663079 30150 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:15:59.595297 29599 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:59.669266 24013 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:15:59.690765 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:15:59.691543 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:59.692165 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:59.692724 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:15:59.683925 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 308202us
I20260709 12:15:59.695307 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.695940 23387 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:15:59.696041 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.696673 24016 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.697152 24016 raft_consensus.cc:2368] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:59.659596 30164 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:59.706918 29844 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.525s	user 0.006s	sys 0.002s
I20260709 12:15:59.707370 29844 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:59.712239 30130 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.533s	user 0.001s	sys 0.006s
I20260709 12:15:59.712605 30156 consensus_queue.cc:1059] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:15:59.659006 30153 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.715265 30153 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.662297 30163 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.719578 30163 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.721413 24011 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:15:59.726245 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.727350 23387 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:15:59.727813 23389 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:59.728725 23362 leader_election.cc:400] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:15:59.729070 23362 leader_election.cc:403] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:15:59.731164 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.731133 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.735702 23467 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:15:59.729030 30026 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:15:59.741261 30026 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.734661 30010 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:59.743676 30026 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.689509 30067 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:59.744161 30027 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.000s
I20260709 12:15:59.743824 30010 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:59.744505 30067 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.746279 30063 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:15:59.746358 30010 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.746883 30067 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.696053 29987 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.747349 23390 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:15:59.657100 30158 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:15:59.712271 30133 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.534s	user 0.002s	sys 0.000s
I20260709 12:15:59.682763 29482 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.755293 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.756153 23312 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:15:59.759856 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.732167 30009 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:15:59.732906 30074 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:15:59.734078 29844 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:59.766296 23461 raft_consensus.cc:1275] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:15:59.766774 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:15:59.774168 23390 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:59.766860 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.775141 23464 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:15:59.774983 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.776214 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.768009 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.768605 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:15:59.769650 29892 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:15:59.773017 23467 pending_rounds.cc:85] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 9
I20260709 12:15:59.779387 23467 pending_rounds.cc:107] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:15:59.779698 23467 raft_consensus.cc:2889] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:15:59.778262 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.780656 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.780890 23461 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:15:59.782358 23461 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:15:59.783253 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.783999 23464 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:15:59.785836 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.790354 30119 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:15:59.790797 30119 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:15:59.791028 30179 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:15:59.791491 30161 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:15:59.791759 30119 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.793143 30119 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:15:59.791522 30179 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:15:59.804020 23461 pending_rounds.cc:85] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 9
I20260709 12:15:59.804507 23461 pending_rounds.cc:107] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:15:59.804911 23461 raft_consensus.cc:2889] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:15:59.794046 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.794189 30147 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.794654 30150 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:15:59.802012 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:15:59.815091 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.818025 23312 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:15:59.818118 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:15:59.826746 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:15:59.828604 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:59.834770 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54551.6us
I20260709 12:15:59.835698 23362 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:15:59.836724 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53436.4us
I20260709 12:15:59.839211 23393 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:15:59.850199 23982 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:15:59.862568 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:15:59.779141 30074 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:15:59.859820 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 99860us
I20260709 12:15:59.834407 30067 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.816969 30164 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:15:59.791818 30176 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:15:59.810017 29844 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 12:15:59.853050 30009 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:15:59.912210 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 76590us
I20260709 12:15:59.913062 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:15:59.914767 30158 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:15:59.915046 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.915544 30179 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:15:59.916481 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:15:59.913780 30015 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:59.921608 30027 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:15:59.921653 29997 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:15:59.924572 29997 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:15:59.925001 29997 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:15:59.857507 30096 consensus_queue.cc:1048] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:15:59.917243 29580 raft_consensus.cc:2804] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:15:59.944145 29580 raft_consensus.cc:697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:15:59.933799 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:15:59.997283 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56609.2us
W20260709 12:15:59.998668 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47580.2us
I20260709 12:15:59.887066 30067 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:00.007159 23361 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:15:59.992156 30015 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:15:59.945135 29580 consensus_queue.cc:237] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.003178 30124 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:00.035133 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:00.035313 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 172214us
I20260709 12:16:00.040060 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:00.045619 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:00.039443 29596 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.035108 30169 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:00.104472 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:00.112351 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 3060) took 1140 ms. Trace:
W20260709 12:16:00.112705 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 67170.1us
I20260709 12:16:00.045521 30185 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.056874 29992 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:00.057103 30179 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:00.106472 30136 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:16:00.113482 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:00.131094 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.112744 23321 rpcz_store.cc:276] 0709 12:15:58.972140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.001866 (+ 29726us) service_pool.cc:224] Handling call
0709 12:15:59.002239 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:59.096581 (+ 94342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:00.112071 (+1015490us) spinlock_profiling.cc:243] Waited 86.6 ms on lock 0x7b54002747d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:00.112291 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199230848}
I20260709 12:16:00.060216 30158 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.064011 29482 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.136029 30185 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:00.148752 30183 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:16:00.052385 30009 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.126825 30063 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:00.156251 30074 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.156392 30166 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.109615 29596 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:00.157701 30192 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:16:00.159108 29596 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.141978 30158 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.162798 30185 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.163793 30158 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.170815 30022 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.171456 30022 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.157187 30166 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.174407 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.175251 23463 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:00.176025 30189 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:16:00.178854 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:00.179600 30195 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:00.156479 30194 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:00.151721 30102 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:16:00.197325 30035 consensus_meta.cc:287] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 1.077s	user 0.012s	sys 0.002s
W20260709 12:16:00.241137 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 62308.6us
I20260709 12:16:00.186582 30092 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.199967 30164 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.248602 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:00.201537 30194 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.253624 30194 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:16:00.257421 30157 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.746s	user 0.003s	sys 0.000s
I20260709 12:16:00.258415 30149 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:00.261996 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 1150) took 1017 ms. Trace:
I20260709 12:16:00.262537 24012 rpcz_store.cc:276] 0709 12:15:59.244553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.244674 (+   121us) service_pool.cc:224] Handling call
0709 12:15:59.245211 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:59.248332 (+  3121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:59.248361 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:59.248872 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:15:59.383421 (+134549us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:59.383466 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:15:59.383539 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:00.256731 (+873192us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:00.261548 (+  4817us) raft_consensus.cc:1685] finished
0709 12:16:00.261582 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:00.261912 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1397,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":25681664,"thread_start_us":134118,"threads_started":1,"wal-append.queue_time_us":134502}
I20260709 12:16:00.274886 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42792 (request call id 2277) took 1306 ms. Trace:
I20260709 12:16:00.275380 23314 rpcz_store.cc:276] 0709 12:15:58.968464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:58.969671 (+  1207us) service_pool.cc:224] Handling call
0709 12:16:00.274689 (+1305018us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b54001a81d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:00.274816 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893872384}
I20260709 12:16:00.280318 30164 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.281765 30067 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.282639 30164 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.241816 30102 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.283413 30102 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.249763 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:00.152230 29997 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.114020 30169 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:00.361245 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 111622us
W20260709 12:16:00.366114 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 64779.9us
I20260709 12:16:00.284610 30194 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.375332 30194 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.376509 30135 raft_consensus.cc:3055] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:00.366006 30121 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:00.382126 30121 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.382483 30121 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:00.382617 29997 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.366343 30022 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.371519 30162 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.247325 30035 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.384907 30162 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.387066 30035 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.388083 24010 raft_consensus.cc:2368] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21.
W20260709 12:16:00.388197 30204 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:16:00.377100 30135 raft_consensus.cc:740] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:00.391088 30135 consensus_queue.cc:260] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.392498 30135 raft_consensus.cc:3060] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:00.396147 30009 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.396733 30009 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.397804 29997 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.398311 29997 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.287346 30166 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.401067 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:00.420408 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 290035us
I20260709 12:16:00.419529 24009 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:00.422305 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:00.440369 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 326975us
I20260709 12:16:00.440316 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.441630 23396 raft_consensus.cc:2410] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:16:00.442574 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.443344 23395 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:00.446523 29992 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:00.447142 29992 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:00.458099 23393 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 3060) took 1287 ms. Trace:
I20260709 12:16:00.458647 23393 rpcz_store.cc:276] 0709 12:15:59.171180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.171326 (+   146us) service_pool.cc:224] Handling call
0709 12:15:59.171656 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:15:59.839630 (+667974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:00.457765 (+618135us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b54001e0fd4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:00.458013 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534848768}
I20260709 12:16:00.462281 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:00.463130 23318 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:00.366653 30164 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.474687 24012 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: []
I20260709 12:16:00.475800 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:00.478003 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 50845.9us
I20260709 12:16:00.422631 29892 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:00.471309 30204 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:16:00.469340 30022 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:00.437819 30153 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.482982 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:00.484118 23318 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:16:00.467260 30142 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.485811 24009 raft_consensus.cc:1240] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
I20260709 12:16:00.488166 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.488844 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:00.488948 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:00.490108 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:00.493990 23312 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:00.494971 30009 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.497707 30142 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:00.481263 30186 log.cc:927] Time spent T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:16:00.491482 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:00.505381 23314 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:16:00.505810 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.491074 23469 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:00.508607 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.509227 23469 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:00.510890 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:00.511456 23392 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:00.472863 30065 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.512181 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.490998 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.497731 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.501418 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.513679 23312 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:00.503548 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:00.513063 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:16:00.514461 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 149804us
I20260709 12:16:00.520975 23466 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:00.522468 23314 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:00.506489 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:00.524175 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:00.524394 23393 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:00.525751 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:00.535209 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.486203 30022 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.536695 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:00.535917 23464 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:00.537274 23390 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:00.542804 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:00.513648 30103 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:16:00.483397 29844 consensus_queue.cc:1059] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:16:00.556210 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:00.534390 30119 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.719s	user 0.013s	sys 0.006s
I20260709 12:16:00.557358 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:00.540454 30022 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.551132 29943 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:00.483479 29892 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.560407 23396 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:16:00.563241 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.561074 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:00.551632 29789 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.492394 30164 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.538650 30035 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:00.563339 23314 raft_consensus.cc:2393] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:00.565760 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.572306 24012 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:00.573057 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.536779 30185 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:00.492480 30153 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.513454 29580 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:16:00.586117 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:00.586764 24011 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:00.560865 30148 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.590127 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.563172 30169 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.568260 30105 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.593130 23470 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 3041) took 1365 ms. Trace:
I20260709 12:16:00.738965 23470 rpcz_store.cc:276] 0709 12:15:59.227293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.227417 (+   124us) service_pool.cc:224] Handling call
0709 12:15:59.227784 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:59.490030 (+262246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:59.490061 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:15:59.522391 (+ 32330us) log.cc:844] Serialized 26 byte log entry
0709 12:15:59.671842 (+149451us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:15:59.671895 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:00.583362 (+911467us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54001879d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:00.583416 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:00.583476 (+    60us) raft_consensus.cc:1685] finished
0709 12:16:00.583489 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:00.593035 (+  9546us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25052,"raft.queue_time_us":212361,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7032,"spinlock_wait_cycles":310396800,"thread_start_us":225390,"threads_started":2,"wal-append.queue_time_us":381755}
I20260709 12:16:00.565704 30142 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:00.688956 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 109947us
W20260709 12:16:00.689716 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 132113us
W20260709 12:16:00.693142 30210 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:16:00.585379 30185 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.690864 30162 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.695034 29892 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.694212 30174 raft_consensus.cc:3055] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:00.790602 30174 raft_consensus.cc:740] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:00.791457 30174 consensus_queue.cc:260] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.791867 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:00.792654 30174 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:00.738840 30158 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:00.793823 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.794480 23464 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:00.751221 30105 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.696022 30178 consensus_queue.cc:1059] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.145s
I20260709 12:16:00.796132 30178 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:00.796476 30178 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:00.796654 30105 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.797339 30178 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.696648 30141 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.799166 30121 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.799261 30141 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.584362 29789 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.759728 30185 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:00.702663 30140 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:16:00.705706 30221 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:16:00.707250 30212 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:00.708256 29997 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.540351 30065 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.715994 30153 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.716214 30193 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:00.721007 23392 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
W20260709 12:16:00.727878 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 147457us
I20260709 12:16:00.561139 30061 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.709103 30164 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.551692 30166 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:16:00.735394 23466 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:00.730724 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 169003us
I20260709 12:16:00.760035 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:16:00.853502 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 93576.9us
I20260709 12:16:00.587121 30119 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.161s
W20260709 12:16:00.748775 30124 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.703s	user 0.016s	sys 0.000s
I20260709 12:16:00.809880 29943 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:00.761301 30169 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.775652 29672 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:00.778663 30163 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:00.871457 23317 raft_consensus.cc:1813] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Handling vote request from an unknown peer 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:00.780527 30022 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.783512 29992 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.786998 30150 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.787969 29899 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:00.791927 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.793701 30158 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.703173 30145 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:00.704638 30211 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:16:00.803179 23467 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.001s	sys 0.012s
I20260709 12:16:00.803797 23319 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:16:00.806486 23470 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:16:00.805852 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55783.5us
I20260709 12:16:00.877060 23463 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
W20260709 12:16:00.882870 23467 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.001s	sys 0.012s
I20260709 12:16:00.837212 30178 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:00.813364 29789 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.813617 30212 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.724071 30148 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:00.889501 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:00.857586 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:00.897938 23982 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:00.865022 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:00.873335 23317 raft_consensus.cc:2368] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:00.856170 30103 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.876298 23388 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
W20260709 12:16:00.862433 30102 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.579s	user 0.014s	sys 0.001s
I20260709 12:16:00.860618 30165 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:00.870198 30226 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.883255 23980 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:00.852983 30153 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:16:00.858455 30061 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.887012 29899 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.853252 30193 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.868227 30149 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:16:00.873596 30169 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.970960 29899 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:00.915241 30156 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:00.971805 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:00.972057 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 60301.8us
I20260709 12:16:00.971807 30156 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.930233 30142 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:00.876358 30163 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.973580 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:00.973721 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:00.974500 30149 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:16:00.915987 30102 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.975095 24011 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:00.974509 24016 raft_consensus.cc:2393] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:00.918962 30141 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.915097 30167 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:00.975215 30149 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:00.887311 30145 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.977056 30149 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.978809 30145 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.979478 30145 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:00.980021 30145 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.980069 30142 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.981176 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:00.981992 23466 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:00.981846 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.982214 30142 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.982928 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.983054 30061 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.919130 30226 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:00.983805 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:00.945904 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62727.5us
I20260709 12:16:00.989217 23312 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:00.986567 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:01.011324 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.011991 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:01.014886 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 107756us
I20260709 12:16:01.041700 23318 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:01.042111 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:01.051863 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:01.052603 23318 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:01.053362 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:01.054005 23470 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:01.054941 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:01.054863 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:01.057972 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:00.912899 30165 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.063112 23316 raft_consensus.cc:2410] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:16:00.902108 29789 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.896166 30212 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:16:00.968853 30199 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:16:01.084085 23390 consensus_meta.cc:287] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.001s
I20260709 12:16:01.084659 23390 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:01.060668 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:00.913173 30035 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:01.075153 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:01.063373 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.095239 23313 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:16:01.076478 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.095963 24007 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 23.
I20260709 12:16:01.077131 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:01.079404 23436 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:00.965755 30140 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:01.081406 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:00.964663 29992 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:00.881768 30150 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.941339 30121 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.962575 30065 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.050011 30193 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:01.014397 30027 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:01.039302 30167 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:00.940521 30022 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:00.975435 30163 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.012579 29899 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:01.073607 30105 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:01.087910 29789 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:01.103569 24010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 1147) took 1950 ms (client timeout 1655 ms). Trace:
I20260709 12:16:01.104585 23320 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:16:01.106693 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 3066) took 1826 ms. Trace:
I20260709 12:16:01.074788 30165 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:01.064160 30142 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:01.092478 24011 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:01.099109 30035 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:01.122668 24010 rpcz_store.cc:269] 0709 12:15:59.153342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.154488 (+  1146us) service_pool.cc:224] Handling call
0709 12:16:01.103297 (+1948809us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54006153d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:01.103456 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2833454976}
I20260709 12:16:01.123436 23317 rpcz_store.cc:276] 0709 12:15:59.280252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.280440 (+   188us) service_pool.cc:224] Handling call
0709 12:16:01.106503 (+1826063us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54003af054. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:01.106636 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3655848192}
I20260709 12:16:00.917843 30103 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.113476 30150 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.134150 30158 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.121937 29992 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 12:16:01.126067 30027 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.143728 30103 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.144647 30027 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.145761 23469 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:01.148313 30022 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.149044 30022 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.159403 29789 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:01.129990 30167 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:16:01.162561 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.161815 30167 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:00.938071 29943 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.125801 30105 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.126273 29580 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.166006 30193 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:01.166383 30035 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:01.130409 30142 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:01.190877 30169 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.199460 30226 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.182673 30009 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.200449 30009 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.203795 30223 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:16:01.204372 30223 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:01.179534 30022 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.179364 30027 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:01.129526 29899 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.182138 30105 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:01.193094 30142 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:01.192584 30145 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.205999 30027 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:01.229463 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.228380 30150 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:01.248608 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:01.249373 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:01.250535 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:01.205406 30223 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.231388 29899 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:01.289883 23470 raft_consensus.cc:2410] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:16:01.226154 30153 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:01.188802 29580 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.227500 30102 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.288172 30027 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:01.225392 30212 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.290802 30166 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:01.228487 30185 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:01.310297 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 249695us
I20260709 12:16:01.292610 30150 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.316270 24009 raft_consensus.cc:2393] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:01.316483 23468 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:01.318883 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.324335 23387 raft_consensus.cc:1275] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:01.326864 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:01.327241 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.329525 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.330044 24010 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:16:01.296635 30153 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.308913 30148 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.327700 23387 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:01.330204 23391 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:01.347347 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.350696 23389 raft_consensus.cc:2410] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:16:01.352247 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:01.325026 30159 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:01.354705 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.355855 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.360021 24016 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:01.311061 30166 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.358213 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.361241 23389 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:16:01.363727 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.368894 23391 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
I20260709 12:16:01.369458 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:01.369885 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.371479 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.373924 24013 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 23.
I20260709 12:16:01.367573 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.364367 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.376468 24016 raft_consensus.cc:2393] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 23: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:01.377089 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.345945 29992 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.345889 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:01.396888 23387 raft_consensus.cc:2468] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 23.
I20260709 12:16:01.399063 30150 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.402324 30009 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.403849 29580 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.410964 23390 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:01.413874 23396 raft_consensus.cc:606] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:01.414444 23396 raft_consensus.cc:614] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:01.421303 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:01.424492 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:01.409945 30235 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:01.426746 30174 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:16:01.429006 30158 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:01.434046 30233 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:16:01.434394 30158 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.436666 30166 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:01.437677 30235 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.440056 30153 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.416299 29992 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:01.422235 30222 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.537s	user 0.005s	sys 0.006s
I20260709 12:16:01.447192 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:01.451915 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.452808 23387 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:16:01.453967 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.449990 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:01.426884 30140 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:01.440238 30235 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:01.467958 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.418846 30165 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.469803 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.471951 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:01.469339 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:01.476963 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.477018 23391 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:01.449895 30102 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:01.738411 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:01.748198 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 247965us
I20260709 12:16:01.750582 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:01.463567 30191 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:01.765761 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:01.821823 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:01.822716 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:01.823568 24011 raft_consensus.cc:2368] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
W20260709 12:16:01.825904 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 75404us
W20260709 12:16:01.843176 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 130467us
I20260709 12:16:01.793038 30236 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:01.758651 29987 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:01.481902 30240 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:01.785966 30239 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.746276 29672 consensus_queue.cc:1048] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.164s
I20260709 12:16:01.830107 30178 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.869731 29943 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:01.784602 30009 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:01.799503 30169 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:01.784472 30022 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:01.877008 30185 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:01.908560 30145 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:01.965952 30234 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.613s	user 0.000s	sys 0.001s
I20260709 12:16:01.997534 30169 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:01.992587 30178 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.014931 30149 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:02.015172 30178 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.015411 30169 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.015797 30178 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:01.997557 30022 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.016383 30178 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.017549 30185 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:02.018486 30246 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:16:02.018355 30185 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.023149 23391 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:16:02.025280 29987 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.025801 29987 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:01.997479 30009 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.026381 30022 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.027917 30009 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.028155 30185 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.035411 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:02.038825 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.044955 23362 leader_election.cc:400] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:16:02.045506 23362 leader_election.cc:403] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:16:02.055861 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.056641 24011 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:02.060017 30165 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.070899 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:01.874434 23982 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:02.052670 30027 consensus_meta.cc:287] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.743s	user 0.012s	sys 0.000s
W20260709 12:16:02.076599 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 202262us
I20260709 12:16:02.077831 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:02.076506 30022 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.080689 30022 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.053422 30169 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:02.002480 30145 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.074019 29672 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:02.081077 30252 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:16:02.080892 23360 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:02.089903 30258 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:02.090794 30253 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:02.090553 30258 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.091243 30022 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.092110 29672 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.093719 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:02.094233 30219 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.060575 30174 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.094969 30219 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.098094 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:02.121666 23392 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:16:01.922721 30236 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.100030 30178 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.094537 29672 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.086081 30102 raft_consensus.cc:2804] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:02.084373 30027 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.967315 30140 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:01.785022 30244 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.341s
I20260709 12:16:02.118078 29992 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:02.096709 30105 consensus_meta.cc:287] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.868s	user 0.013s	sys 0.000s
I20260709 12:16:02.106211 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2875) took 2337 ms. Trace:
I20260709 12:16:01.992065 30239 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.074576 30009 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.126761 30225 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:02.111318 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2286) took 1060 ms. Trace:
W20260709 12:16:02.099619 30247 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:16:02.127907 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.156147 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.093073 30066 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.129258 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.138895 23392 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:02.101080 29987 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.127002 30022 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:02.147476 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.120298 30240 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.094312 23389 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:02.156420 24008 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:02.164348 23395 raft_consensus.cc:2368] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 23. Current term is 24.
I20260709 12:16:02.164460 23469 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2891) took 1179 ms. Trace:
I20260709 12:16:02.160241 24011 raft_consensus.cc:2410] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:16:02.159613 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.148286 23461 rpcz_store.cc:276] 0709 12:15:59.769100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.803297 (+ 34197us) service_pool.cc:224] Handling call
0709 12:15:59.803598 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:15:59.818320 (+ 14722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:59.818386 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.181738 (+1363352us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b2c0011e398. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:16:01.182142 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.421674 (+239532us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:01.421953 (+   279us) log.cc:844] Serialized 14 byte log entry
0709 12:16:01.426942 (+  4989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.427047 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.102220 (+675173us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:02.105968 (+  3748us) raft_consensus.cc:1685] finished
0709 12:16:02.105993 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.106152 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":712,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":77,"raft.queue_time_us":4761,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5833,"spinlock_wait_cycles":14454912,"thread_start_us":239209,"threads_started":1,"wal-append.queue_time_us":821010}
I20260709 12:16:02.152954 23318 rpcz_store.cc:276] 0709 12:16:01.053703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.055141 (+  1438us) service_pool.cc:224] Handling call
0709 12:16:01.112406 (+ 57265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.112732 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.112760 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.113235 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.315498 (+202263us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:01.315542 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.315627 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.029588 (+713961us) raft_consensus.cc:1685] finished
0709 12:16:02.029614 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.110971 (+ 81357us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b54002418d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.111236 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2684,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7435,"spinlock_wait_cycles":130976000,"thread_start_us":201842,"threads_started":1,"wal-append.queue_time_us":334891}
I20260709 12:16:02.165601 23392 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:02.163606 24016 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:02.128281 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.122359 30174 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.166735 24013 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:16:02.159673 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:02.160219 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.164925 23469 rpcz_store.cc:276] 0709 12:16:00.981040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.983574 (+  2534us) service_pool.cc:224] Handling call
0709 12:16:02.160585 (+1177011us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.160710 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370196352}
I20260709 12:16:02.141875 30015 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.148427 30102 raft_consensus.cc:697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:02.178404 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:02.181133 30155 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.183372 30258 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.183665 30155 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.185988 30155 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.181320 30149 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.186914 30149 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.188563 30219 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.188977 30149 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.189855 30145 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.158283 30009 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.192241 30009 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.195065 29987 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:02.195642 30035 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.984s	user 0.006s	sys 0.003s
I20260709 12:16:02.157222 30105 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.198411 30105 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.166402 30022 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.199237 30022 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:02.202343 30102 consensus_queue.cc:237] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.182686 30015 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.206933 30236 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.208292 30015 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.196072 30035 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.214913 30027 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.217334 30174 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:02.227327 23466 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1507) took 1148 ms. Trace:
I20260709 12:16:02.227353 24015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53926 (request call id 1358) took 1220 ms. Trace:
I20260709 12:16:02.228487 24015 rpcz_store.cc:276] 0709 12:16:01.032004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.050261 (+ 18257us) service_pool.cc:224] Handling call
0709 12:16:01.092201 (+ 41940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.092420 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.092447 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.092868 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.427209 (+334341us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:01.427243 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.427319 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.227087 (+799768us) raft_consensus.cc:1685] finished
0709 12:16:02.227109 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.227252 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":256040,"threads_started":1,"wal-append.queue_time_us":1043157}
I20260709 12:16:02.181658 30165 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:02.230293 30165 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:02.227798 23466 rpcz_store.cc:276] 0709 12:16:01.079164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.118406 (+ 39242us) service_pool.cc:224] Handling call
0709 12:16:01.146683 (+ 28277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:01.147141 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:01.147180 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:01.147841 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.350340 (+202499us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:01.350377 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:01.350436 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.098719 (+748283us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:02.227003 (+128284us) raft_consensus.cc:1685] finished
0709 12:16:02.227029 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.227255 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13185,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":144,"spinlock_wait_cycles":23691520,"thread_start_us":202084,"threads_started":1,"wal-append.queue_time_us":599701}
I20260709 12:16:02.231400 30165 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:02.232950 23982 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.194:41721 timed out after 1.531s (SENT) [suppressed 18 similar messages]
I20260709 12:16:02.233943 23389 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
W20260709 12:16:02.234467 30142 consensus_meta.cc:287] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.946s	user 0.008s	sys 0.004s
I20260709 12:16:02.237778 30142 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.239830 30142 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.241176 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.242798 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.246681 23389 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:02.234546 30035 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.162360 30066 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.221215 30022 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:02.271099 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:02.272434 23388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 1226) took 1776 ms (client timeout 1531 ms). Trace:
I20260709 12:16:02.228022 30239 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:02.272966 23388 rpcz_store.cc:269] 0709 12:16:00.489828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.490540 (+   712us) service_pool.cc:224] Handling call
0709 12:16:02.256987 (+1766447us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001e3f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.266365 (+  9378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834598656}
I20260709 12:16:02.287580 30236 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.288259 30236 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.290480 30236 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.292812 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.293627 23466 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:16:02.173812 30240 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.296154 23466 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:16:02.296680 30240 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.298743 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.299515 23466 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:02.301931 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1881) took 1790 ms. Trace:
I20260709 12:16:02.302341 23319 rpcz_store.cc:276] 0709 12:16:00.511579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.511700 (+   121us) service_pool.cc:224] Handling call
0709 12:16:00.534519 (+ 22819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:00.884949 (+350430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.301700 (+1416751us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400148f54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:02.301874 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618560768}
W20260709 12:16:02.304819 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 174748us
W20260709 12:16:02.324174 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.327293 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.330577 23469 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:16:02.332063 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:02.334190 23360 leader_election.cc:400] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:02.343762 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.344518 23461 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:02.304137 30169 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19.
I20260709 12:16:02.305838 30236 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.358631 30209 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:02.358853 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.359052 30236 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.359730 30209 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:02.331087 30156 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.348943 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36042 (request call id 3051) took 1560 ms. Trace:
I20260709 12:16:02.360170 30209 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:02.357751 30155 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.351562 30105 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.351780 30149 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.357729 30219 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.391903 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2428) took 1074 ms. Trace:
I20260709 12:16:02.392334 23390 rpcz_store.cc:276] 0709 12:16:01.317815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.317953 (+   138us) service_pool.cc:224] Handling call
0709 12:16:01.410688 (+ 92735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:01.411543 (+   855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.391697 (+980154us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b54003453d0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.391844 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146199424}
I20260709 12:16:02.331375 30225 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:02.350121 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.396454 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.325700 30070 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.363236 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.401371 23316 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:02.360453 23465 rpcz_store.cc:276] 0709 12:16:00.788988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.789099 (+   111us) service_pool.cc:224] Handling call
0709 12:16:00.789384 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:00.802209 (+ 12825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:00.802241 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:00.802689 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:16:01.180236 (+377547us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:01.180300 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.329627 (+1149327us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b540044b454. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:02.329675 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.329718 (+    43us) raft_consensus.cc:1685] finished
0709 12:16:02.329732 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.348856 (+ 19124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":912,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8896,"spinlock_wait_cycles":28637440,"thread_start_us":377198,"threads_started":1,"wal-append.queue_time_us":508749}
I20260709 12:16:02.388613 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.358240 30066 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.394402 30236 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.361536 30155 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.394503 30075 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.397050 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:02.397821 23393 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 1228) took 1894 ms (client timeout 2240 ms). Trace:
I20260709 12:16:02.328521 30162 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:02.402779 23317 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:02.404585 30156 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.412688 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.395191 30159 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.405180 30229 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:16:02.417294 30265 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
W20260709 12:16:02.424250 23393 rpcz_store.cc:269] 0709 12:16:00.503529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.505580 (+  2051us) service_pool.cc:224] Handling call
0709 12:16:02.397664 (+1892084us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b540005ad54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.397767 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41261440}
I20260709 12:16:02.426329 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.428843 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:02.428943 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 196014us
I20260709 12:16:02.438506 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.439662 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.352656 30262 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.421811 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.435936 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.453215 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.453883 23393 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:02.403604 30254 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.458014 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.396091 30105 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.450085 23319 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:16:02.361127 30209 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.427809 30236 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.427349 30185 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:02.460016 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.429046 30075 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.394446 30149 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.462767 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.461567 30066 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.431352 30156 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.400710 30219 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.473613 24009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 1162) took 1609 ms. Trace:
I20260709 12:16:02.452265 30159 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:02.481118 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.419103 30176 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:02.430661 30162 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.472954 30116 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:02.487219 23319 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:16:02.491771 30218 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 1.907s	user 0.010s	sys 0.018s
W20260709 12:16:02.493745 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 62639.1us
I20260709 12:16:02.501541 23436 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.494422 24009 rpcz_store.cc:276] 0709 12:16:00.864599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.864758 (+   159us) service_pool.cc:224] Handling call
0709 12:16:02.473402 (+1608644us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54005613d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.473557 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953898368}
I20260709 12:16:02.419426 30070 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.476972 30185 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:02.463757 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44018.7us
I20260709 12:16:02.504556 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:45800 (request call id 56) took 1181 ms. Trace:
I20260709 12:16:02.504621 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.509827 30155 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.510980 30105 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.513543 23396 rpcz_store.cc:276] 0709 12:16:01.328649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.329875 (+  1226us) service_pool.cc:224] Handling call
0709 12:16:02.495952 (+1166077us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54003453d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a966e853 00007ff0ae63c60a 00007ff0a7bc27d4 00007ff0a7bc2762 00007ff0a7bc2691 00007ff0a7bc260c 00007ff0a7bc18c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.496116 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52019968}
I20260709 12:16:02.511492 30149 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.514865 30185 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:02.519210 30219 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:02.520392 30267 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:02.523022 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2424) took 2083 ms. Trace:
I20260709 12:16:02.523381 23394 rpcz_store.cc:276] 0709 12:16:00.439321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.439449 (+   128us) service_pool.cc:224] Handling call
0709 12:16:00.439751 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:00.469719 (+ 29968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:00.469748 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:00.470220 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:16:02.153580 (+1683360us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b180032a100. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:16:02.399523 (+245943us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:02.399570 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.399674 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.522701 (+123027us) raft_consensus.cc:1685] finished
0709 12:16:02.522720 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.522957 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51098,"raft.queue_time_us":37092,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":1997,"spinlock_wait_cycles":3419904,"thread_start_us":245736,"threads_started":1,"wal-append.queue_time_us":1987810}
I20260709 12:16:02.500166 30116 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.525749 30116 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:16:02.515148 30193 consensus_meta.cc:287] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.326s	user 0.015s	sys 0.000s
I20260709 12:16:02.529819 23317 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:02.529860 30193 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:02.530848 30193 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.530977 30162 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.531157 30272 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:02.531725 30272 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.532172 30272 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:02.533841 30075 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.531293 30193 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.494251 30066 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.455438 30262 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.536130 30193 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.536466 30066 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.537061 30262 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.537604 30022 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.540527 30185 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.466341 30254 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.543298 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.478085 30236 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.554661 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.558547 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.559123 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.559235 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.569509 24016 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.559748 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.560570 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:02.562510 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:02.570523 24014 raft_consensus.cc:2368] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18.
I20260709 12:16:02.563181 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.550143 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:02.541411 30116 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.573885 24009 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 13: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:02.575698 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.576821 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.569773 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.553303 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.578191 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.580035 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.580896 23393 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:02.586858 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.587836 23362 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.588891 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.591281 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.591742 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.591905 23464 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:02.592872 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.593468 24015 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:02.595065 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:02.590982 30155 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:02.597944 30141 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:02.598500 30141 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.598685 30236 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.598888 30141 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:02.601693 30257 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:02.552073 30140 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.598008 30155 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.596374 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:02.604158 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.623102 23461 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:02.604382 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.605362 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:02.624992 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 71753.7us
I20260709 12:16:02.625569 23463 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:16:02.614660 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:02.626040 23436 leader_election.cc:400] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 23. Current term is 24.
I20260709 12:16:02.626444 23436 leader_election.cc:403] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:02.627126 23464 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:16:02.616673 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:02.627987 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:02.628752 23470 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:02.573467 30193 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:02.629616 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:02.501394 30176 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.630369 23462 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:02.593390 30239 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:02.630877 23393 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:02.631938 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.632449 23388 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:02.633404 23396 raft_consensus.cc:2410] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:16:02.638574 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:02.643679 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.655725 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.656579 23390 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:02.605331 30257 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:02.660445 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:02.633291 30236 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:02.636384 30105 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.626899 30274 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:02.676961 30070 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.627800 30140 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.654493 30261 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:16:02.679363 30254 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.677372 30274 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 12:16:02.677851 30236 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:02.680719 30254 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:02.623049 30263 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:16:02.668128 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.630695 30193 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.628229 30169 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:02.633903 30239 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.658042 30063 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:02.683928 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:16:02.654829 30271 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:16:02.595933 30022 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:16:02.689671 24016 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
I20260709 12:16:02.751395 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:02.752188 24016 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:16:02.699625 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.706040 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.709231 23393 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:02.755601 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.673058 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.731443 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.667867 30096 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:16:02.737955 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:02.745131 30262 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.745023 30070 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:02.747188 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 120115us
I20260709 12:16:02.768599 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.768754 23436 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:02.770254 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.770805 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:02.674788 30257 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:02.726398 30193 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:02.761644 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.763167 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.691841 30263 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:02.734484 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 106583us
I20260709 12:16:02.732643 30239 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:02.736191 30063 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.803740 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.817332 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.765267 30194 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.677608 30105 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.807032 30070 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.805794 30263 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:02.895538 30070 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.828552 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.896718 24015 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:02.822499 30275 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:02.898257 30275 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:16:02.901867 30263 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.904131 30263 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.909533 23463 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
I20260709 12:16:02.913930 30172 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.914134 30075 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 23. Current term is 24.
I20260709 12:16:02.914714 30172 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.917524 30075 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.918270 30075 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.920571 30229 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:02.924618 30223 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:02.838840 23467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2868) took 3069 ms. Trace:
I20260709 12:16:02.819264 30063 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:02.817111 30155 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.762392 30096 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:02.938861 30096 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:02.835145 29789 consensus_queue.cc:1059] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:16:02.940209 29789 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.940677 29789 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.921257 30229 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:02.842612 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.844516 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.952719 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:02.953538 23462 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:02.963377 30105 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.850948 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52270 (request call id 2438) took 1099 ms. Trace:
I20260709 12:16:02.859351 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.968308 23393 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:02.861333 29990 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:02.864701 30212 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.865909 30272 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:02.838042 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.872113 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:02.825546 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.838016 30176 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.867630 30156 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.883659 30140 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.885727 30159 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:02.681572 30236 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.839088 23468 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60318 (request call id 1503) took 2303 ms. Trace:
I20260709 12:16:02.995380 23468 rpcz_store.cc:276] 0709 12:16:00.535504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:00.535666 (+   162us) service_pool.cc:224] Handling call
0709 12:16:00.554313 (+ 18647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:01.316913 (+762600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.838842 (+1521929us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54003edad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:02.839028 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752366464}
I20260709 12:16:02.889106 30185 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.890149 30116 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.890812 30194 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.911782 30158 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:02.937945 23467 rpcz_store.cc:276] 0709 12:15:59.764484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:59.764633 (+   149us) service_pool.cc:224] Handling call
0709 12:15:59.764960 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:15:59.801510 (+ 36550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:15:59.801556 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:15:59.802202 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 12:16:00.034983 (+232781us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:16:00.885654 (+850671us) log.cc:844] Serialized 14 byte log entry
0709 12:16:01.321348 (+435694us) log.cc:844] Serialized 14 byte log entry
0709 12:16:01.321601 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.833339 (+1511738us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b54002251d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:02.833416 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:02.833612 (+   196us) raft_consensus.cc:1685] finished
0709 12:16:02.833632 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:02.833863 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24301,"prepare.queue_time_us":71061,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":87,"raft.queue_time_us":214247,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":11012,"spinlock_wait_cycles":17855360,"thread_start_us":1387714,"threads_started":3,"wal-append.queue_time_us":433660,"wal-append.run_cpu_time_us":4823,"wal-append.run_wall_time_us":2160049}
I20260709 12:16:02.972709 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.014822 23392 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:02.973122 23396 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:03.019699 24014 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:03.020915 30194 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:02.979612 29990 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.022648 29990 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:03.029953 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:02.967201 23391 rpcz_store.cc:276] 0709 12:16:01.765120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.765358 (+   238us) service_pool.cc:224] Handling call
0709 12:16:02.850687 (+1085329us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400344ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:02.850867 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512630144}
I20260709 12:16:03.031857 30015 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:02.983282 24009 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:02.982792 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:03.033248 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50582us
I20260709 12:16:03.033941 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:03.035225 30165 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:03.035609 30165 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:03.035971 30165 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.037077 30035 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:02.981220 30172 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.043196 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.030546 29943 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:03.032325 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:02.984045 23362 leader_election.cc:400] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18.
I20260709 12:16:03.047221 23362 leader_election.cc:403] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:16:03.047298 23389 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
W20260709 12:16:03.048048 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 70402.5us
I20260709 12:16:03.048926 23360 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:03.049295 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:02.803131 30262 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:02.998713 23393 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:16:03.045883 23318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.094s	sys 0.138s
I20260709 12:16:02.819552 24016 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:03.012374 30209 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.055405 23362 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:03.057497 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.057348 23360 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.076496 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.079887 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.081389 23312 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:16:03.061110 24007 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:03.081511 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.085021 23321 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:16:03.054330 23318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.094s	sys 0.138s
I20260709 12:16:03.092047 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.052073 30015 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 17. Current term is 18.
I20260709 12:16:03.102948 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.052635 30035 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:03.044618 30141 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.053853 29997 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:03.089802 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:03.056126 30263 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:03.107254 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49981.6us
I20260709 12:16:03.129051 23321 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:03.055671 30226 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:03.119762 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.095453 30022 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:03.053954 30283 raft_consensus.cc:3055] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:03.094141 30027 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:03.118412 30245 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:03.117712 30262 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.095188 30214 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:03.106956 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.155755 23312 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:03.134255 23360 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.132987 23462 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:16:03.157850 23362 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:03.156374 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:03.190100 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55938.9us
W20260709 12:16:03.164764 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.191805 23360 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.122360 29997 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:03.099357 30102 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:16:03.054419 29943 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.151388 30092 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:16:03.192600 23313 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:03.151367 29992 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:16:03.197101 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 426330us
I20260709 12:16:03.192071 30022 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:03.151620 30015 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:16:03.254282 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.209120 23312 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:03.130685 30226 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:03.247133 30227 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:16:03.264357 30280 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:16:03.238906 30035 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.273947 30227 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.259711 29992 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.274946 29992 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:03.270262 30244 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:16:03.276249 30165 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.276890 30257 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.284186 30165 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:03.272275 30063 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.160524 30027 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.200398 30065 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:16:03.233001 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 442496us
I20260709 12:16:03.279951 30239 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.202263 30142 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:16:03.310015 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47158.5us
I20260709 12:16:03.311481 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.268942 30022 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.162381 30272 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:03.265165 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2439) took 1476 ms. Trace:
I20260709 12:16:03.281443 30102 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:03.282064 30226 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.285202 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.315371 23466 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:03.286134 30257 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.313140 23387 rpcz_store.cc:276] 0709 12:16:01.788784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:01.879349 (+ 90565us) service_pool.cc:224] Handling call
0709 12:16:01.973694 (+ 94345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:02.162642 (+188948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.162674 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:02.169696 (+  7022us) log.cc:844] Serialized 52 byte log entry
0709 12:16:02.353704 (+184008us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:16:02.867347 (+513643us) spinlock_profiling.cc:243] Waited 969 us on lock 0x7b1800146f20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0ad06c3c7 00007ff0a967e9a7 00007ff0a968e0bf 00007ff0a968e038
0709 12:16:03.200630 (+333283us) log.cc:844] Serialized 14 byte log entry
0709 12:16:03.246906 (+ 46276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.257214 (+ 10308us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:03.257290 (+    76us) raft_consensus.cc:1685] finished
0709 12:16:03.264972 (+  7682us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.265109 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45841,"raft.queue_time_us":101906,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":4448,"spinlock_wait_cycles":2227968,"thread_start_us":640135,"threads_started":3,"wal-append.queue_time_us":166003}
I20260709 12:16:03.134589 30283 raft_consensus.cc:740] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:03.288376 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.196071 29997 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.299503 23394 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:03.300143 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:03.301652 30063 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.303574 30065 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.304491 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:03.305990 30140 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:03.344965 30140 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:03.345825 30140 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.312077 23462 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.319772 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.347854 23469 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.320694 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.334488 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.348963 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.343572 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:03.350219 23466 raft_consensus.cc:2468] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
W20260709 12:16:03.350690 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 46271.2us
I20260709 12:16:03.352938 23470 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:03.355014 23462 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:03.357008 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.358103 23462 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:03.360117 23394 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:03.361333 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.362035 23466 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 8.
I20260709 12:16:03.363204 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.332908 30142 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:03.364430 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.368136 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.368921 23467 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:03.303108 29990 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.372300 23396 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:03.340189 30164 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:03.405694 30164 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.364289 30212 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:16:03.407810 30164 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.347463 30065 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:03.409744 30240 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:03.410254 30240 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.412266 30240 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.414788 30075 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.415249 30283 consensus_queue.cc:260] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.416373 30283 raft_consensus.cc:3060] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:03.416641 30214 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.417052 30214 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.365386 30257 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:03.418262 30257 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:03.419178 30257 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.347247 30140 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:03.373872 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.373872 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.427470 23463 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:03.371178 30015 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:03.429662 24010 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 11: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:03.429904 30015 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.432008 30015 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.379489 30258 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.432534 30227 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.432731 30258 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.433137 30165 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.433712 30165 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.435220 30258 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.436902 23463 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:03.437593 23320 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 13.
I20260709 12:16:03.437991 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.439252 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.439728 23463 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.439762 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.440932 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.441852 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.442173 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:03.444525 30299 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:03.445729 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.446997 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:03.451241 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.452046 23470 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:03.451287 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.453346 23321 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 21: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:03.382395 30239 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.382702 29943 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.389403 30141 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.490649 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:03.395941 30282 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:16:03.402303 30142 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.494534 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:03.423835 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 51654us
I20260709 12:16:03.425225 23469 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:03.457280 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.458914 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.496445 23362 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:03.459524 23387 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:16:03.460810 23392 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:16:03.481796 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.488788 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:03.391934 30193 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.574s	user 0.009s	sys 0.001s
I20260709 12:16:03.491463 23461 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:16:03.414789 30027 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.493165 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.505456 23316 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:03.497643 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:03.491572 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 390701us
I20260709 12:16:03.506079 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.408998 30229 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.491468 23315 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
W20260709 12:16:03.508074 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50863.7us
I20260709 12:16:03.498765 23463 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:03.512271 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:03.513911 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.517010 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.461755 30102 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:03.529134 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.533986 23469 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:03.534875 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.535454 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:03.529352 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.529706 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.537086 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.537719 23470 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:16:03.532286 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.538932 23321 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:03.535084 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.547134 23392 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:03.547871 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.549834 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.550607 23469 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.552388 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.554183 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.555328 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.558012 23463 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:03.504768 30239 raft_consensus.cc:2804] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:03.559463 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:03.502075 30172 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:03.529359 30096 consensus_meta.cc:287] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.004s
I20260709 12:16:03.566256 23390 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:03.569265 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1479) took 1014 ms. Trace:
I20260709 12:16:03.567652 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.571712 23393 raft_consensus.cc:2393] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 21: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:03.566898 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:03.576162 23470 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:03.577961 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:03.578725 23470 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:03.588502 23470 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:03.589887 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.532786 30141 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:16:03.569839 24011 rpcz_store.cc:276] 0709 12:16:02.555401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.555583 (+   182us) service_pool.cc:224] Handling call
0709 12:16:02.556175 (+   592us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:02.556647 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.556684 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:02.587065 (+ 30381us) log.cc:844] Serialized 26 byte log entry
0709 12:16:02.736760 (+149695us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:02.736796 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:02.736859 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:03.568964 (+832105us) raft_consensus.cc:1685] finished
0709 12:16:03.568995 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.569188 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17439,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":4588,"thread_start_us":104557,"threads_started":1,"wal-append.queue_time_us":556604}
I20260709 12:16:03.590688 23470 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:03.533344 30027 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:03.565482 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:03.590627 24012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53940 (request call id 1164) took 1028 ms. Trace:
I20260709 12:16:03.533897 30286 raft_consensus.cc:2764] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:16:03.608762 23469 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
W20260709 12:16:03.611254 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 346257us
W20260709 12:16:03.616581 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 51219.1us
I20260709 12:16:03.507189 30193 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.618696 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:03.605226 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.619383 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.624297 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.598888 30172 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.634956 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.612488 23980 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.621299 23463 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:03.647248 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.605393 30239 raft_consensus.cc:697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:03.612968 30260 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:03.687542 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.546929 30102 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.702102 23469 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:03.705175 23469 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:03.712070 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.714365 23465 raft_consensus.cc:2368] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21.
W20260709 12:16:03.716331 30271 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 12:16:03.716401 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:03.718011 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.718804 24016 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:16:03.692471 23316 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:03.606132 30141 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:03.696324 23980 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:03.610716 30231 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:03.599239 30096 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.629922 30165 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:03.642571 30027 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:16:03.693128 23287 connection.cc:256] RPC call timeout handler was delayed by -1.1727472320001198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 12:16:03.748803 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0384484010000961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 12:16:03.748878 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52625.7us
I20260709 12:16:03.629727 30258 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:03.749519 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0384484010000961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
W20260709 12:16:03.755043 23287 connection.cc:256] RPC call timeout handler was delayed by -1.0224472320001041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 12:16:03.755407 23287 connection.cc:256] RPC call timeout handler was delayed by -1.008748401000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
W20260709 12:16:03.755704 23287 connection.cc:256] RPC call timeout handler was delayed by -1.008748401000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1592s for a response
I20260709 12:16:03.642007 30169 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:16:03.743791 30194 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.010s	sys 0.036s
I20260709 12:16:03.731396 29949 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:03.745319 30141 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:16:03.766898 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 298808us
I20260709 12:16:03.761469 23394 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:03.765100 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.775971 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.722123 30281 raft_consensus.cc:2764] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:16:03.776585 23316 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.774626 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:16:03.769192 30194 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.011s	sys 0.036s
I20260709 12:16:03.762056 30174 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:03.760377 30169 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:03.787988 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.791831 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.780736 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.792971 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.792246 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.778103 29949 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:03.778769 30178 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:03.799782 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.801947 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.767987 30248 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 12:16:03.789815 29992 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.003s
I20260709 12:16:03.785655 30276 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:03.803530 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.803758 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:03.804344 23393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:03.762673 30258 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.804607 23316 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:03.790007 30174 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:03.811249 23393 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:03.794849 30169 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.834405 23393 raft_consensus.cc:614] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:03.834914 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:03.806254 30209 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.743s	user 0.009s	sys 0.003s
I20260709 12:16:03.844681 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.806272 30178 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.832257 30223 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:03.857810 29949 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.859853 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:03.852401 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:03.831127 30276 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.808561 30229 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:03.852662 24013 raft_consensus.cc:2410] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:16:03.853026 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.770784 30239 consensus_queue.cc:237] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:03.837131 30174 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:03.875828 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:03.853528 30178 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:03.866614 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:03.883077 24016 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:03.830406 29992 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.855720 30223 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:03.876968 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120685us
I20260709 12:16:03.802273 30027 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.886754 23462 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:16:03.866003 30276 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.893384 30102 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:03.904599 29997 consensus_meta.cc:287] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.000s
I20260709 12:16:03.891575 23980 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.913697 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.914144 30258 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.905136 29997 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.915385 30102 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:03.915803 30102 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.916168 30102 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:03.916234 23391 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
W20260709 12:16:03.920441 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 140294us
I20260709 12:16:03.921315 30193 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.921774 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.921231 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.922433 23319 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
W20260709 12:16:03.923555 30293 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:16:03.924052 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:03.891690 30223 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:03.907090 29789 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.895010 30096 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:03.939194 23362 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:03.892442 30065 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:03.982337 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:03.983031 23321 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:03.982388 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:03.987684 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:03.988461 23320 raft_consensus.cc:3060] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:03.987104 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:04.010193 23396 raft_consensus.cc:1240] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:03.984201 29997 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:03.986661 30214 consensus_meta.cc:287] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 12:16:03.993863 30258 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.010634 30035 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:03.990285 30297 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:16:04.036422 30295 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:16:04.038414 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 1888) took 1685 ms. Trace:
I20260709 12:16:04.039167 23320 raft_consensus.cc:2468] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:04.029179 30214 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.042114 30214 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:04.043056 30305 log.cc:927] Time spent T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:16:04.046099 30301 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 12:16:04.047880 30298 log.cc:927] Time spent T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
W20260709 12:16:04.049082 30310 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:16:04.055146 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.026728 30311 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:04.056649 30311 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.057014 30311 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:04.038941 23318 rpcz_store.cc:276] 0709 12:16:02.352499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.354912 (+  2413us) service_pool.cc:224] Handling call
0709 12:16:02.355161 (+   249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:03.091686 (+736525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:03.091726 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:03.102588 (+ 10862us) log.cc:844] Serialized 26 byte log entry
0709 12:16:03.549791 (+447203us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:16:03.549845 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.779363 (+229518us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000ec254. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:03.779419 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:04.038007 (+258588us) raft_consensus.cc:1685] finished
0709 12:16:04.038038 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.038332 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40480,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":280070528,"thread_start_us":974023,"threads_started":2,"wal-append.queue_time_us":644145}
I20260709 12:16:04.075388 30209 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:04.075038 30178 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.075989 30209 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.078099 30178 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.078128 30209 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:04.080219 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 87915.3us
I20260709 12:16:04.085973 29992 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:04.031083 30035 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.095656 23465 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36038 (request call id 2907) took 1500 ms. Trace:
I20260709 12:16:04.096057 23465 rpcz_store.cc:276] 0709 12:16:02.594747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.594850 (+   103us) service_pool.cc:224] Handling call
0709 12:16:04.090805 (+1495955us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400230ad4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:04.095589 (+  4784us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541802240}
I20260709 12:16:04.091634 30035 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.104312 30065 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.108980 29990 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.110628 24013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53940 (request call id 1163) took 1550 ms. Trace:
I20260709 12:16:04.111099 24013 rpcz_store.cc:276] 0709 12:16:02.560580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.569531 (+  8951us) service_pool.cc:224] Handling call
0709 12:16:04.110421 (+1540890us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540028f0d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:04.110554 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2930060672}
W20260709 12:16:04.112782 30226 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.770s	user 0.008s	sys 0.004s
I20260709 12:16:04.111516 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:04.117231 29949 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:04.121233 30297 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:04.121718 30297 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:04.121572 29949 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.122143 30297 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:04.123857 29949 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:04.117139 30226 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.132504 30226 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.133100 30226 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:04.135613 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 196510us
I20260709 12:16:04.139063 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:04.136145 30229 raft_consensus.cc:2804] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:04.142469 30142 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.143043 30142 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:04.144837 30300 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260709 12:16:04.147642 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.148063 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:04.149718 30165 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:04.150236 30165 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:04.151223 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:04.151110 30165 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.153781 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.155453 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.156476 30229 raft_consensus.cc:697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:04.158831 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:04.160188 23362 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:04.160929 30065 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:04.118489 30283 consensus_meta.cc:287] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.001s
I20260709 12:16:04.161578 30065 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:04.162297 23470 raft_consensus.cc:1275] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:16:04.119217 30276 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.157425 30229 consensus_queue.cc:237] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.162253 30065 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:04.162161 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:04.176723 23391 raft_consensus.cc:606] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:04.176914 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:04.177258 23391 raft_consensus.cc:614] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:04.166514 30311 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.132848 30141 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.175974 30209 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:03.604789 24012 rpcz_store.cc:276] 0709 12:16:02.562091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.573481 (+ 11390us) service_pool.cc:224] Handling call
0709 12:16:02.573786 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:02.574004 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:16:03.052461 (+478457us) log.cc:844] Serialized 14 byte log entry
0709 12:16:03.589404 (+536943us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:03.590329 (+   925us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:03.590385 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.590457 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:03.590572 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28497,"prepare.queue_time_us":23558,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":30147,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7925,"thread_start_us":964197,"threads_started":2}
W20260709 12:16:04.184952 30140 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.000s
I20260709 12:16:04.186707 30102 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.190399 30169 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:04.186450 30291 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.466s	user 0.002s	sys 0.000s
I20260709 12:16:04.176803 30226 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.165369 30276 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.211633 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:04.229838 23391 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:04.230250 23391 raft_consensus.cc:614] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:16:04.213315 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 289318us
I20260709 12:16:04.194785 23360 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:04.145220 30142 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:04.240278 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.242385 23393 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:04.195293 30022 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:16:04.222898 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51482.2us
W20260709 12:16:04.225030 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 56524.6us
I20260709 12:16:04.118000 30258 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:04.237347 23387 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
W20260709 12:16:04.209635 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 88636.6us
I20260709 12:16:04.246469 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:04.271359 23391 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:04.250728 30174 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.274468 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:04.193994 30065 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:04.282975 23464 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2914) took 1332 ms. Trace:
I20260709 12:16:04.204164 30172 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.245196 30227 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.293514 23362 leader_election.cc:400] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21.
W20260709 12:16:04.258359 30292 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.561s	user 0.001s	sys 0.001s
I20260709 12:16:04.259351 30169 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:04.260102 30311 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.314819 30065 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.316887 29949 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:04.261785 23388 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:04.263540 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.191475 30260 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:04.265229 23316 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:16:04.275846 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.278573 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:04.283361 23464 rpcz_store.cc:276] 0709 12:16:02.950485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:02.964271 (+ 13786us) service_pool.cc:224] Handling call
0709 12:16:02.964760 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:02.965204 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:02.965246 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:02.965914 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 12:16:03.318508 (+352594us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:03.318548 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:03.318614 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:04.185179 (+866565us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:04.264173 (+ 78994us) raft_consensus.cc:1685] finished
0709 12:16:04.264188 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:04.282917 (+ 18729us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1698,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":31509,"spinlock_wait_cycles":89534848,"thread_start_us":352241,"threads_started":1,"wal-append.queue_time_us":646609}
I20260709 12:16:04.288245 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:04.289698 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.299937 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:04.277521 30158 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:16:04.342721 30272 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:04.342914 23387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:04.343189 30272 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.343506 23387 raft_consensus.cc:606] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:04.343987 23387 raft_consensus.cc:614] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:04.303807 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:04.304519 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:04.345147 23466 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:16:04.346319 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.347054 24016 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:04.310859 30102 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:04.309010 30226 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.351611 23217 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 11 to 12, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.310847 30309 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:04.311348 30027 consensus_queue.cc:1059] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:04.322289 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.327710 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.354557 24011 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:04.327626 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:04.328321 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.265488 30022 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:04.356653 30022 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:04.356819 30307 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:04.357620 30307 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.360272 30307 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:04.360935 30307 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.336423 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:16:04.336160 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48009.3us
I20260709 12:16:04.361759 23320 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:04.362787 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:04.364301 30158 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:04.333112 23316 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:04.365099 30258 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.365722 23316 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.367203 23316 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.336738 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:04.335930 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.369128 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:04.343556 30272 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:04.344780 23468 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 21: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:04.305907 30141 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:04.310004 23215 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 320) took 16458 ms (client timeout 15000 ms). Trace:
I20260709 12:16:04.341267 24014 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:16:04.371240 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:04.392015 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:04.335176 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46020.6us
I20260709 12:16:04.394717 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.395471 23321 raft_consensus.cc:2368] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
I20260709 12:16:04.396193 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.397148 24016 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:04.397724 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.356654 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 11 to 13, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 13 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.399713 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.400791 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.381156 23466 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:16:04.383172 23320 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:04.397195 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.402462 23463 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:04.402848 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.403383 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:04.404207 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.404378 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.398706 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.408664 23319 raft_consensus.cc:2393] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 21: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:04.409789 23316 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:16:04.411047 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.410961 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.332413 30320 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:04.413041 30320 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.414829 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.416169 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:04.418206 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.419898 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.422326 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.423707 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.424963 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.337327 30227 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:04.430272 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.413228 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.290512 30140 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.108s
I20260709 12:16:04.442333 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:04.447197 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:04.317158 30297 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.451843 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:04.453385 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 47770.9us
I20260709 12:16:04.454917 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.465710 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
W20260709 12:16:04.385278 23215 rpcz_store.cc:269] 0709 12:15:47.856522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:47.863751 (+  7229us) service_pool.cc:224] Handling call
0709 12:16:04.309904 (+16446153us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:15:58.542626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:15:58.543278 (+   652us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:15:58.543321 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:15:58.543346 (+    25us) tablet.cc:662] Decoding operations
0709 12:15:58.622525 (+ 79179us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:15:58.629777 (+  7252us) write_op.cc:641] Partition lock acquired for write op
0709 12:15:58.634634 (+  4857us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:15:58.800901 (+166267us) tablet.cc:701] Row locks acquired
0709 12:15:58.800924 (+    23us) write_op.cc:260] PREPARE: finished
0709 12:15:58.801074 (+   150us) write_op.cc:270] Start()
0709 12:15:58.801190 (+   116us) write_op.cc:276] Timestamp: P: 1783599358801024 usec, L: 0
0709 12:15:58.801211 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:15:58.801963 (+   752us) log.cc:844] Serialized 9528 byte log entry
0709 12:15:59.686231 (+884268us) op_driver.cc:464] REPLICATION: finished
0709 12:15:59.868206 (+181975us) write_op.cc:301] APPLY: starting
0709 12:15:59.868297 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:00.264647 (+396350us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:00.264764 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:00.776626 (+511862us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:00.786215 (+  9589us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 12:16:00.786275 (+    60us) write_op.cc:312] APPLY: finished
0709 12:16:00.786806 (+   531us) log.cc:844] Serialized 212 byte log entry
0709 12:16:01.192185 (+405379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:01.228173 (+ 35988us) write_op.cc:454] Released schema lock
0709 12:16:01.236134 (+  7961us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:01.316008 (+ 79874us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:01.326007 (+  9999us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1561666,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":175436,"apply.run_cpu_time_us":98725,"apply.run_wall_time_us":1438884,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104543,"num_ops":24,"prepare.queue_time_us":443323,"prepare.run_cpu_time_us":33557,"prepare.run_wall_time_us":490357,"raft.queue_time_us":236699,"raft.run_cpu_time_us":24865,"raft.run_wall_time_us":169836,"replication_time_us":873653,"thread_start_us":1141191,"threads_started":5,"wal-append.queue_time_us":853990,"wal-append.run_cpu_time_us":42150,"wal-append.run_wall_time_us":2717566}]]}
I20260709 12:16:04.468345 23321 raft_consensus.cc:2368] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:16:04.471760 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.473868 23217 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 14, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 14 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.483445 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.485265 23468 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:16:04.485692 23464 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:04.487610 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.490291 23313 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
W20260709 12:16:04.549742 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.549587 23217 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 14 to 15, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 15 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.451486 30015 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:04.568992 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.416109 30214 consensus_queue.cc:1048] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:16:04.385068 30307 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.416764 30320 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:04.571702 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.415845 30172 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.124s
W20260709 12:16:04.584105 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 183358us
W20260709 12:16:04.585467 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 102102us
I20260709 12:16:04.585338 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.586943 23321 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:04.587179 23362 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.588264 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.588666 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:04.482195 30283 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:04.577582 23313 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:16:04.592916 30316 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:16:04.597203 30096 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:16:04.549594 30260 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.597148 30035 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.598045 30260 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.600800 30035 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:04.601202 29943 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:04.552981 30302 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:04.606578 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.609378 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.612725 24014 raft_consensus.cc:2368] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:16:04.614042 30257 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:04.614538 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.615391 23318 raft_consensus.cc:2368] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17.
W20260709 12:16:04.616552 30324 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:16:04.617514 30302 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.617885 30231 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:04.601408 30260 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:04.619728 30302 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:04.621706 30322 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:16:04.583204 30015 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.653534 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.645485 23360 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:04.649493 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.649111 23217 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 13, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 13 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:04.657318 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:04.658375 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 193958us
W20260709 12:16:04.662015 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 257729us
I20260709 12:16:04.592865 30317 consensus_queue.cc:1059] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:16:04.671314 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.670351 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.680805 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.607040 30283 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.687705 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:04.691110 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.691903 23318 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:04.692620 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.656769 30096 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:04.673620 30323 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:16:04.694304 23217 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 7 to 8, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 8 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.702785 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.697757 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.670886 30231 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:16:04.686719 30223 consensus_meta.cc:287] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.000s
I20260709 12:16:04.711463 30168 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:16:04.729718 24010 raft_consensus.cc:1240] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:04.730753 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.731369 24013 raft_consensus.cc:1240] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:04.674636 30263 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:04.670449 30169 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:04.734617 23217 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:04.773710 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76066.7us
I20260709 12:16:04.778393 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.778496 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.642951 30227 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:04.710753 30174 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:04.781327 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.801733 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.803233 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:04.784881 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:04.787626 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.806425 23217 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.808667 24016 raft_consensus.cc:2368] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 21. Current term is 22.
W20260709 12:16:04.810482 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.700977 30035 consensus_queue.cc:1059] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 12:16:04.774402 30158 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.781863 30096 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:04.871755 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.876621 24012 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 21: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:04.873374 23217 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 16 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 18 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:04.779567 30244 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:04.819540 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.944725 30158 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:04.950423 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 340512us
I20260709 12:16:04.986761 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:04.949415 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:04.775516 30223 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.877802 24010 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:04.806509 30319 raft_consensus.cc:2764] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:16:04.799089 30169 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:16:04.948820 23434 leader_election.cc:336] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.632s (SENT)
W20260709 12:16:04.886230 30239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.020s
I20260709 12:16:04.992468 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:04.816450 29992 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.147s
I20260709 12:16:04.994084 23434 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:04.993875 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:04.995270 23217 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:05.000550 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:05.003340 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.008848 23462 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:16:04.783663 30263 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:04.948444 30027 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.010749 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:05.016243 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:05.025792 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:05.026579 23395 raft_consensus.cc:3060] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:05.029914 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:05.030467 23393 raft_consensus.cc:2393] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 18: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:05.030645 23468 raft_consensus.cc:3060] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:05.035079 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.031445 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:05.011798 30300 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:16:05.037703 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:05.038456 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:04.990088 30276 raft_consensus.cc:2764] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:16:05.039620 23315 raft_consensus.cc:2393] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:05.044411 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.045059 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:04.987192 30297 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:05.019464 30239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.002s	sys 0.020s
I20260709 12:16:05.060616 23469 raft_consensus.cc:2393] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:05.008698 30065 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:16:04.948221 30096 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:05.068265 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 265122us
I20260709 12:16:05.061280 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:05.069176 23390 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:16:05.121539 23469 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:05.122881 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:05.110451 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:05.027733 30168 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:16:05.128149 23469 raft_consensus.cc:1240] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:16:05.139029 23468 raft_consensus.cc:2468] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:05.140357 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:05.147570 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:04.995383 30231 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.163226 24013 raft_consensus.cc:1240] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:05.168496 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:05.047394 30027 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.178743 30361 raft_consensus.cc:2764] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:16:05.181108 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:05.182382 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:16:05.172101 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 141566us
W20260709 12:16:05.193496 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 62751.3us
I20260709 12:16:05.152709 30258 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.204891 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:05.173692 30307 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.186519 30283 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:05.180145 30361 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:16:05.208175 23392 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:16:05.403398 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 198527us
I20260709 12:16:05.409590 23436 leader_election.cc:400] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 21. Current term is 22.
I20260709 12:16:05.411083 23436 leader_election.cc:403] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:05.199222 30015 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.236605 30174 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.415365 30174 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.294239 30276 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:05.418358 30096 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.418386 30276 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:05.419528 30276 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:05.421422 30158 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.297436 30223 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.170540 30169 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:05.298076 30283 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:05.293852 30311 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:16:05.248148 30334 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.369s	user 0.000s	sys 0.000s
I20260709 12:16:05.432317 30212 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:05.432960 30212 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:16:05.208807 30335 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 12:16:05.308559 30340 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.181s	user 0.002s	sys 0.002s
W20260709 12:16:05.245399 30343 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:16:05.434347 30214 raft_consensus.cc:3055] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:05.297760 30348 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:05.353721 30356 consensus_queue.cc:1059] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:16:05.441821 30349 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:05.440275 30223 consensus_queue.cc:1059] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:05.306447 30361 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:05.439996 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43553.4us
W20260709 12:16:05.299186 30354 log.cc:927] Time spent T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.093s	user 0.000s	sys 0.003s
I20260709 12:16:05.447947 30168 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:05.444661 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2468) took 1044 ms. Trace:
I20260709 12:16:05.467823 30214 raft_consensus.cc:740] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:05.487528 30361 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:16:05.493525 30283 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 21. Current term is 22.
W20260709 12:16:05.496735 30022 consensus_meta.cc:287] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 1.139s	user 0.014s	sys 0.015s
W20260709 12:16:05.492620 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.497255 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53950 (request call id 1500) took 1010 ms. Trace:
I20260709 12:16:05.507584 30168 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.513979 30348 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.522265 30356 raft_consensus.cc:3055] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:05.522536 30174 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.523937 30212 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.528371 23982 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:05.530489 30027 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.530946 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.531577 23312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2320) took 1060 ms. Trace:
I20260709 12:16:05.532397 23468 raft_consensus.cc:2393] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 14: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:05.508210 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 99059.6us
I20260709 12:16:05.540668 23396 raft_consensus.cc:1240] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:05.503717 24014 rpcz_store.cc:276] 0709 12:16:04.486891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.487369 (+   478us) service_pool.cc:224] Handling call
0709 12:16:05.496708 (+1009339us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54006301d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:05.496977 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263177984}
I20260709 12:16:05.522945 30356 raft_consensus.cc:740] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:05.544863 30360 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:16:05.480664 23387 rpcz_store.cc:276] 0709 12:16:04.400457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.401634 (+  1177us) service_pool.cc:224] Handling call
0709 12:16:04.436474 (+ 34840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:04.482648 (+ 46174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:04.482676 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:04.483093 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 12:16:04.674486 (+191393us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:04.674528 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.674728 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:05.443969 (+769241us) raft_consensus.cc:1685] finished
0709 12:16:05.444014 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:05.444397 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84625,"raft.queue_time_us":117187,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":14260,"thread_start_us":191107,"threads_started":1,"wal-append.queue_time_us":315640}
I20260709 12:16:05.532091 23312 rpcz_store.cc:276] 0709 12:16:04.471016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.477498 (+  6482us) service_pool.cc:224] Handling call
0709 12:16:04.479379 (+  1881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:04.479787 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:04.479828 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:04.480366 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:16:04.784256 (+303890us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:04.870591 (+ 86335us) log.cc:844] Serialized 14 byte log entry
0709 12:16:05.061493 (+190902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.061582 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:05.530928 (+469346us) raft_consensus.cc:1685] finished
0709 12:16:05.530967 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:05.531264 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89158,"prepare.run_cpu_time_us":4757,"prepare.run_wall_time_us":64803,"raft.queue_time_us":119991,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":80397,"thread_start_us":390795,"threads_started":2,"wal-append.queue_time_us":468014}
I20260709 12:16:05.556253 30254 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:05.556851 30254 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:05.558605 30254 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:05.560106 30254 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:05.549988 30214 consensus_queue.cc:260] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.566677 30311 raft_consensus.cc:3055] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:05.567320 30311 raft_consensus.cc:740] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:05.578872 23315 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 3132) took 1431 ms. Trace:
I20260709 12:16:05.579360 23315 rpcz_store.cc:276] 0709 12:16:04.147217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.147374 (+   157us) service_pool.cc:224] Handling call
0709 12:16:05.578409 (+1431035us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54003c0854. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:05.578656 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036218624}
I20260709 12:16:05.580665 29992 raft_consensus.cc:2804] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:16:05.584056 30344 log.cc:927] Time spent T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.288s	user 0.005s	sys 0.000s
I20260709 12:16:05.591277 30226 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.592269 30226 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.598196 23436 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:05.600047 23395 consensus_meta.cc:287] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.004s
I20260709 12:16:05.606320 23395 raft_consensus.cc:2468] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
W20260709 12:16:05.609632 30333 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.616s	user 0.002s	sys 0.000s
I20260709 12:16:05.610672 30168 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.601572 29992 raft_consensus.cc:697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:05.497241 30022 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:05.634203 23391 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:05.633787 23468 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:16:05.635386 30022 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.618549 29992 consensus_queue.cc:237] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.614346 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:05.637526 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 95616.7us
I20260709 12:16:05.637799 30223 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:05.638553 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:05.620484 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 76374.7us
W20260709 12:16:05.620905 30345 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
W20260709 12:16:05.643590 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.644217 30317 raft_consensus.cc:3055] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:05.644654 30317 raft_consensus.cc:740] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:05.644907 23320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42792 (request call id 2316) took 1225 ms. Trace:
I20260709 12:16:05.646190 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:05.646939 30245 raft_consensus.cc:3055] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:05.647588 30245 raft_consensus.cc:740] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:05.646637 23320 rpcz_store.cc:276] 0709 12:16:04.419855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.464801 (+ 44946us) service_pool.cc:224] Handling call
0709 12:16:04.465111 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:04.465396 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:04.465421 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:04.465902 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:16:04.874896 (+408994us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:04.874934 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:04.875008 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:05.619600 (+744592us) spinlock_profiling.cc:243] Waited 4.73 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:05.644403 (+ 24803us) raft_consensus.cc:1685] finished
0709 12:16:05.644433 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:05.644720 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10870144,"thread_start_us":325575,"threads_started":1,"wal-append.queue_time_us":731249}
I20260709 12:16:05.621134 30283 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.649787 30356 consensus_queue.cc:260] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.651803 30356 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:05.627548 23982 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:05.613925 30348 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.627866 30102 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:05.568923 30311 consensus_queue.cc:260] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:05.656793 30311 raft_consensus.cc:3060] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:05.657594 30102 raft_consensus.cc:2804] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:05.658340 30102 raft_consensus.cc:697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:05.649145 30245 consensus_queue.cc:260] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.663582 30245 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:05.663429 30102 consensus_queue.cc:237] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:05.628899 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.657616 30212 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:05.707723 30317 consensus_queue.cc:260] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.708369 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.709339 30317 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:05.711942 30348 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:05.713470 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 764503us
I20260709 12:16:05.713251 30348 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.715656 23466 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
W20260709 12:16:05.716123 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.716645 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.718413 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2467) took 1430 ms. Trace:
I20260709 12:16:05.720929 23396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2461) took 1485 ms. Trace:
I20260709 12:16:05.719103 23391 rpcz_store.cc:276] 0709 12:16:04.288441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.288566 (+   125us) service_pool.cc:224] Handling call
0709 12:16:04.288849 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:05.634678 (+1345829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.717941 (+ 83263us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400196ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:05.718299 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3093189504}
I20260709 12:16:05.716637 30214 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:05.721395 23396 rpcz_store.cc:276] 0709 12:16:04.235671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.235788 (+   117us) service_pool.cc:224] Handling call
0709 12:16:04.236079 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:05.547909 (+1311830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.720615 (+172706us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003458d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:05.720861 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2986517376}
I20260709 12:16:05.727209 23465 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:16:05.723320 30272 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 1.313s	user 0.005s	sys 0.018s
I20260709 12:16:05.723645 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.736136 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:05.736904 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.737165 23321 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
W20260709 12:16:05.742626 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.744311 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.697942 30223 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:05.693213 30361 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.755191 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:05.756062 30361 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.746450 30263 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.750475 30272 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.761931 23463 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:16:05.765573 23314 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:16:05.739498 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:05.736181 23469 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
W20260709 12:16:05.772243 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.741119 30226 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.783442 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:05.785557 30172 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:16:05.787091 23464 raft_consensus.cc:2393] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 18: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:05.752310 30283 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.759763 30263 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:05.801158 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:05.802790 30299 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:05.803481 30299 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.805711 30299 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.806912 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:05.808749 30194 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:16:05.809760 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 171408us
W20260709 12:16:05.841408 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:05.871927 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 83087.1us
W20260709 12:16:05.873134 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 55356.7us
I20260709 12:16:05.840584 23314 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:16:05.882207 23312 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:16:05.884131 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:05.888913 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.883841 30357 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:05.895195 30337 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.895046 30357 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.896091 30337 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.897740 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:05.897728 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 80237.1us
I20260709 12:16:05.902632 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:05.905680 30223 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:05.907749 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:05.909762 30297 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:05.910620 30297 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:16:05.919744 30320 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 1.326s	user 0.015s	sys 0.011s
I20260709 12:16:05.905076 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:05.922870 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 189685us
I20260709 12:16:05.950812 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:05.881321 30263 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.909495 30357 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:05.916486 30337 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:05.952224 23434 leader_election.cc:336] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.584s (SENT)
I20260709 12:16:05.946727 30348 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:05.922521 30283 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.974961 30365 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:16:05.978991 30360 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:05.981856 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:05.982579 30299 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:05.982882 23320 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:05.983234 30299 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:05.972035 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 58254.3us
I20260709 12:16:05.939815 30320 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:05.967866 30254 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
W20260709 12:16:05.996317 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.990017 30337 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:06.000383 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:05.998826 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.004457 23362 leader_election.cc:400] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:16:05.973881 30272 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.004776 23362 leader_election.cc:403] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:16:06.004139 30169 consensus_queue.cc:1059] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:16:06.006616 23316 raft_consensus.cc:2368] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:16:06.008235 23318 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 8. Current term is 9. Ops: []
W20260709 12:16:06.011102 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 106171us
W20260709 12:16:06.016149 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 76198us
I20260709 12:16:06.022756 30320 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:06.023228 23316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 3142) took 1616 ms (client timeout 1982 ms). Trace:
W20260709 12:16:06.029825 23316 rpcz_store.cc:269] 0709 12:16:04.406629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.410603 (+  3974us) service_pool.cc:224] Handling call
0709 12:16:06.023005 (+1612402us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002751d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:06.023159 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3665486464}
W20260709 12:16:06.031488 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 61581.1us
I20260709 12:16:06.071063 23468 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:16:06.072862 23464 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:16:06.073796 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.075999 24014 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:06.032122 30226 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:06.083899 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:06.084052 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.085192 23465 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:06.088951 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.039629 30260 raft_consensus.cc:3055] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:16:06.030526 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 121079us
I20260709 12:16:06.092496 23468 raft_consensus.cc:1275] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:16:06.093444 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.094563 30212 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.092679 30260 raft_consensus.cc:740] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:06.095396 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.044663 30302 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:06.026459 30263 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.014279 30299 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:06.031093 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.680s (SENT)
I20260709 12:16:06.098686 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:06.099926 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 68866.4us
I20260709 12:16:06.024755 30283 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.042296 30358 consensus_queue.cc:1059] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:16:06.101867 30283 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:16:06.102312 30358 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:16:06.102782 30358 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:06.105549 30358 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.107916 30358 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:16:06.122893 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.131850 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.131400 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.097538 30260 consensus_queue.cc:260] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:06.140746 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.141840 30272 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.142442 30272 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.145860 30341 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.146140 30212 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:16:06.146729 30341 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.148525 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.149111 30302 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.149796 30302 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.152030 30302 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:06.091197 30276 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.670s	user 0.017s	sys 0.009s
I20260709 12:16:06.152490 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.168454 23315 raft_consensus.cc:1275] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:16:06.150664 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 55270.3us
I20260709 12:16:06.158092 30320 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:06.173522 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.169086 30272 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:06.179826 30341 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:06.176930 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64294.7us
I20260709 12:16:06.157668 23468 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:16:06.195387 23396 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:06.197000 24014 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:06.185935 30226 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:16:06.185976 30276 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.221379 23317 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:06.216843 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.222182 23387 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:06.226374 23464 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:06.226868 23464 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:06.227898 23464 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.211457 30260 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:06.217932 23360 leader_election.cc:400] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 13 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 13. Current term is 14.
I20260709 12:16:06.218041 24010 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:06.229089 23464 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:16:06.216305 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.247958 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.251513 23387 raft_consensus.cc:1240] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:06.258527 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1266) took 1261 ms. Trace:
I20260709 12:16:06.259008 23389 rpcz_store.cc:276] 0709 12:16:04.997192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.997599 (+   407us) service_pool.cc:224] Handling call
0709 12:16:05.039168 (+ 41569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:05.669203 (+630035us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:05.669260 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:05.669884 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:16:05.964934 (+295050us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:05.964969 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:05.967563 (+  2594us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:06.258173 (+290610us) raft_consensus.cc:1685] finished
0709 12:16:06.258196 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:06.258362 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11083,"raft.queue_time_us":73486,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9902,"thread_start_us":746876,"threads_started":2,"wal-append.queue_time_us":586196}
I20260709 12:16:06.177030 30263 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:16:06.262004 30035 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.262619 30035 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.264870 30035 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
W20260709 12:16:06.268627 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 50925.5us
I20260709 12:16:06.274385 23389 raft_consensus.cc:1240] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:06.274688 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:06.275475 23315 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:06.275578 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.276268 23312 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:06.277066 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.278491 23319 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:16:06.278198 30263 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:06.278883 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.277330 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:06.279645 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 179024us
I20260709 12:16:06.282083 23314 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:06.286018 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.283931 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.295648 30226 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:06.298146 23315 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
W20260709 12:16:06.299470 23434 leader_election.cc:336] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.745s (SENT)
I20260709 12:16:06.306059 23434 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:06.307368 30272 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:06.309279 23434 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.763s (SENT)
I20260709 12:16:06.310002 30276 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:06.310333 23434 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.311874 30272 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:06.312772 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:06.313393 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 77848us
I20260709 12:16:06.314392 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.314780 30276 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.315173 30317 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:06.319181 23434 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.896s (SENT)
I20260709 12:16:06.320101 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:06.321583 30272 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:06.329957 30283 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:06.331403 30283 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:06.331799 30283 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:06.332763 30283 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.335188 30283 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:06.314410 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:06.346625 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.349819 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.352607 23389 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:06.296998 30297 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.350389 23464 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:16:06.353696 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 76716.8us
I20260709 12:16:06.361124 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:06.359776 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43483.1us
I20260709 12:16:06.364800 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.365634 23387 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
W20260709 12:16:06.367744 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48607.5us
I20260709 12:16:06.369282 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.369580 30302 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
W20260709 12:16:06.372849 30387 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 12:16:06.372931 30302 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:16:06.373390 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:06.372772 30245 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:16:06.377565 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.379215 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:06.381563 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.382627 30260 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:06.384480 30272 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:06.379842 30263 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:06.385741 30263 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:06.391038 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 76744.9us
I20260709 12:16:06.358085 30320 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.390096 30263 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:06.420586 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.421142 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.423475 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:06.427843 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.432999 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.436420 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.436949 23468 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:06.417029 30212 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:16:06.451227 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.450621 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
W20260709 12:16:06.464179 30330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.052s
I20260709 12:16:06.438892 30169 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:16:06.467105 23465 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:16:06.467792 23466 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:16:06.460510 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.471609 23463 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:06.457228 30168 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:06.456916 30142 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:16:06.480321 30260 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.480881 30337 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:06.467105 30330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.011s	sys 0.052s
I20260709 12:16:06.473770 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:06.491568 30390 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:16:06.470587 30226 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.481964 30168 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:06.500797 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.465071 30302 consensus_queue.cc:1048] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:16:06.448827 30320 consensus_queue.cc:1059] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:16:06.505228 30168 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:06.515020 30320 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:06.515514 30320 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:06.508476 30276 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:06.503789 30169 consensus_queue.cc:1048] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 12:16:06.502103 30337 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:06.527856 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 83061.9us
I20260709 12:16:06.521131 30320 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:06.544214 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.524814 30245 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:16:06.568049 23434 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.873s (SENT)
W20260709 12:16:06.592679 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.594156 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:06.564635 30226 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:06.609552 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.615186 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:06.613364 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46161.6us
I20260709 12:16:06.624837 30257 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:06.625500 30257 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.627768 30257 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.624886 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.611416 30320 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:06.619248 30272 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:06.614555 30226 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:06.638906 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70887.4us
I20260709 12:16:06.650797 30263 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.654150 30263 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.647744 30276 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:06.660367 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 178319us
I20260709 12:16:06.669905 30142 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:16:06.684252 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:06.697700 23434 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.063s (SENT)
I20260709 12:16:06.698233 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:06.699535 23434 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.139s (SENT)
I20260709 12:16:06.699954 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:06.688059 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 228225us
I20260709 12:16:06.709797 30272 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:06.710884 30283 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:16:06.711151 23434 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.142s (SENT)
I20260709 12:16:06.711674 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:06.712112 30394 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:16:06.712973 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.158s (SENT)
I20260709 12:16:06.677989 30263 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:16:06.713410 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.716785 23360 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.724556 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:06.729931 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.190s (SENT)
I20260709 12:16:06.730423 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.730814 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.687160 30341 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:16:06.722487 30272 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:06.741685 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.743012 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.718703 30283 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:06.726029 30337 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:06.749799 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.206s (SENT)
W20260709 12:16:06.750154 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 125542us
I20260709 12:16:06.751206 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:06.745489 30142 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:16:06.755904 30226 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.748813 30272 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:06.758055 30401 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:06.758719 30401 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:06.759899 30403 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:06.761178 30330 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:06.761658 30330 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:06.757927 30179 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:16:06.757834 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.744063 30260 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:16:06.762933 30330 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.763633 24010 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:06.759917 30401 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.770779 30330 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:06.779068 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:06.787415 30179 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:06.787875 30179 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.788201 30179 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:06.798830 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53211.5us
W20260709 12:16:06.795713 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 78972.8us
I20260709 12:16:06.798969 23362 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:06.799659 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:06.809657 30035 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:16:06.812793 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.802529 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.818246 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:06.822675 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.823532 23317 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:06.815948 30400 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:06.825408 30257 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:06.825592 30400 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:06.825959 30257 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:06.826476 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:06.826642 30400 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.827255 30168 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.828639 30400 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:06.830188 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.831956 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.826423 30257 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:06.832310 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:06.832800 30405 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:06.834077 30405 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:06.834491 30172 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:06.834913 30172 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.835263 30172 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:06.836290 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.802296 30409 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.809207 30403 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:16:06.839890 23979 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.839754 30179 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:06.845580 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.144s (SENT)
I20260709 12:16:06.847568 30179 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:06.843832 30168 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:06.844009 30403 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.849113 30403 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:06.849269 30320 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.851287 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.802028 30297 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:16:06.862973 30257 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.801834 30263 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:16:06.870137 30406 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:06.873858 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 55739.8us
I20260709 12:16:06.876744 24014 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:16:06.876188 30403 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:06.877283 30229 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:06.878561 30403 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:06.876240 30302 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:06.882153 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:06.888922 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.890290 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:06.890654 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.887264 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.890913 23465 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:06.893548 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.876334 30260 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:06.892853 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.891264 24014 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:06.893481 23466 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:06.899092 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:06.899739 23461 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:06.900218 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.901371 30320 raft_consensus.cc:2749] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:06.901705 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.902724 30311 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:06.903079 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:06.899747 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:06.905164 24016 raft_consensus.cc:2393] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:06.910575 23312 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:06.910422 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:06.908713 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:06.911010 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:06.911317 23320 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:06.936625 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 49439.5us
W20260709 12:16:06.943995 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.904913 30212 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:06.953734 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50742.3us
I20260709 12:16:06.946360 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:06.956745 23467 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:16:06.929849 30254 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:06.960880 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.961841 23387 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:06.953998 30226 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.910182 30260 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:06.973901 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:06.972127 23318 raft_consensus.cc:1275] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:16:06.952031 30406 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:06.947105 30297 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:16:06.988653 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:07.019474 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.964182 30409 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:07.025123 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60439.8us
W20260709 12:16:07.025025 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.120s (SENT)
I20260709 12:16:07.029111 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:06.953585 30408 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 12:16:07.016156 30254 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:07.041329 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.041242 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.046797 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:06.971206 30226 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:07.049146 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:07.052103 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:06.956252 30172 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.078171 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:07.069815 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:07.027284 30317 consensus_queue.cc:1048] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:16:07.144317 30102 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.167809 23463 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:16:07.070062 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:07.047776 30356 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:07.145812 30396 log.cc:927] Time spent T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:16:07.050530 30263 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:07.188974 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 163977us
I20260709 12:16:07.025060 30409 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.213619 30403 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.206977 30179 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:07.214201 30179 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.214656 30179 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:07.214265 30403 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:07.238907 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 160840us
I20260709 12:16:07.047610 30254 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:07.243453 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 265016us
I20260709 12:16:07.190937 30257 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.250270 23387 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:07.201567 30102 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.261240 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:07.237943 30419 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:07.303977 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.209553 30263 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.323616 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.327147 23394 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:16:07.327397 30419 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.313552 30229 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.330080 30229 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:07.331768 30414 log.cc:927] Time spent T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
W20260709 12:16:07.333091 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.313964 30276 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:07.356181 23313 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
W20260709 12:16:07.351895 23434 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.585s (SENT) [suppressed 23 similar messages]
W20260709 12:16:07.377115 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 333155us
I20260709 12:16:07.401391 23394 raft_consensus.cc:1275] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:16:07.401598 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 463677us
I20260709 12:16:07.402904 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:07.404553 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:07.415153 23461 consensus_meta.cc:287] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.515s	user 0.011s	sys 0.000s
I20260709 12:16:07.415997 23461 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
W20260709 12:16:07.401098 30416 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.423s	user 0.002s	sys 0.000s
I20260709 12:16:07.344198 30260 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:07.433061 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 171901us
W20260709 12:16:07.447043 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.447533 30272 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:16:07.464908 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:07.468549 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:07.476647 30337 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.495440 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:07.480281 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.495726 30337 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.482363 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.497489 23314 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:07.498122 30337 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:07.498745 30035 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:07.499200 30035 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:07.499619 30035 raft_consensus.cc:3060] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:07.457263 30403 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.505501 30403 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:07.515087 23394 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:16:07.535377 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:16:07.535969 30423 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:16:07.536228 23392 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:07.537464 30426 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:16:07.538383 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.539428 23462 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:16:07.540390 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.540944 23462 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:07.544943 30254 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:07.514642 30226 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.548813 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:07.551192 23467 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 12:16:07.552187 23467 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:16:07.552264 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.554219 30419 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.548619 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:07.559262 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.536137 30417 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:16:07.576762 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 224879us
I20260709 12:16:07.580884 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.581216 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:07.578796 30400 consensus_meta.cc:287] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.002s
I20260709 12:16:07.577551 30299 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.609630 30299 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.612128 30299 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.612686 30299 consensus_queue.cc:1048] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:07.614086 30102 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.578066 30229 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:07.615023 30415 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.399s	user 0.001s	sys 0.000s
I20260709 12:16:07.618127 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.618729 23321 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 9.
I20260709 12:16:07.615818 30179 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.600867 23313 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:07.622383 30179 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.622820 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:07.623476 30172 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.625335 23392 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
W20260709 12:16:07.625818 30424 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:16:07.628634 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:07.629662 23320 consensus_meta.cc:287] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.700s	user 0.003s	sys 0.007s
I20260709 12:16:07.630201 23320 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:16:07.590873 30276 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:07.578205 30417 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:07.639905 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 235410us
I20260709 12:16:07.590546 30448 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:07.643112 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 112477us
W20260709 12:16:07.658300 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.640314 30417 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:07.673641 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.668545 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:07.733615 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 150346us
I20260709 12:16:07.737293 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:07.737118 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.738500 23467 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 8.
I20260709 12:16:07.675212 30283 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:07.749662 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.750231 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:07.747378 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.752641 23467 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:07.753665 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:07.742271 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 85596.3us
I20260709 12:16:07.751847 24012 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:07.732372 30276 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:07.754173 23467 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:07.759639 30427 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:16:07.743250 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:07.732760 30172 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:07.767071 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:07.769819 30452 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:07.769928 30179 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:07.770493 30179 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.770877 30179 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:07.750723 30272 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:07.768555 30448 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:07.773777 30429 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:16:07.779354 30272 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:07.598918 30428 consensus_queue.cc:1059] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.179s
I20260709 12:16:07.798228 30317 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.798874 30317 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.799924 30276 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:07.770483 30452 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:07.800954 30276 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.803200 30452 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:07.804555 30276 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.810711 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:07.811093 30440 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.261s	user 0.003s	sys 0.000s
I20260709 12:16:07.811472 23314 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:07.756198 30283 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.822268 30283 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.825018 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:07.825695 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:07.826210 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:07.827097 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:07.828691 24013 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:07.829874 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:07.832588 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.832864 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.833658 23465 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:16:07.832939 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.838095 23469 raft_consensus.cc:2393] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:07.842336 30263 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.845031 30409 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.846423 23465 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 12.
I20260709 12:16:07.847016 30179 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.849625 30179 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.853703 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:16:07.850224 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 79125.2us
I20260709 12:16:07.858222 24013 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:07.863031 23467 raft_consensus.cc:2468] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:07.801105 30317 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.776270 30172 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:07.863420 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.876250 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.877897 23982 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.881232 30462 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.881850 30462 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:07.882661 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 63687.5us
W20260709 12:16:07.903348 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 239002us
W20260709 12:16:07.910346 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:07.906298 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:07.918715 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:07.888170 30463 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:07.884199 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:07.911535 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:07.888741 30436 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.932336 30436 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:07.933950 30272 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:07.891659 30102 raft_consensus.cc:2804] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:07.936940 30276 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:07.892673 30462 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.885840 30297 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:16:07.872722 30035 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.937656 30179 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:07.938800 30401 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:07.938645 30417 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:07.939719 30438 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:07.939599 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:07.941515 30461 log.cc:927] Time spent T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:16:07.941916 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:07.944787 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:07.946739 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 68934.9us
I20260709 12:16:07.974879 23436 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.970503 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:07.921607 30463 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.977008 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:07.956777 23320 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:16:07.977819 23465 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:07.957118 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.958148 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:07.958513 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:07.930949 30441 log.cc:927] Time spent T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:16:07.921612 30409 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:07.961898 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:07.965621 30179 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.962385 23391 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:07.961898 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:07.984272 23394 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:07.962750 30436 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:07.963327 30297 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.981609 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.990338 23467 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:07.984735 23389 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 20: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:07.982466 23387 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:07.988204 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:07.968804 30401 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.995376 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:07.979496 30272 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:07.962956 30276 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.004339 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:08.004842 30276 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:07.991607 24016 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:08.000566 30317 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:07.979329 30463 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:07.985486 30179 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.001639 30272 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:08.002223 30263 raft_consensus.cc:2764] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:16:08.002421 30035 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:07.998437 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:07.997676 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:07.993551 30297 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:08.001770 23461 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36038 (request call id 2976) took 1048 ms. Trace:
I20260709 12:16:07.989508 30102 raft_consensus.cc:697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.009770 30462 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:08.011137 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.015053 23436 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:07.993402 30438 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:07.995844 30401 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:07.993090 30409 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.024415 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.029808 23461 rpcz_store.cc:276] 0709 12:16:06.958572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:07.425707 (+467135us) service_pool.cc:224] Handling call
0709 12:16:07.426024 (+   317us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:07.626769 (+200745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:07.626799 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:07.642887 (+ 16088us) log.cc:844] Serialized 52 byte log entry
0709 12:16:07.748547 (+105660us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:07.748590 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:07.748667 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:07.940777 (+192110us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:08.001483 (+ 60706us) raft_consensus.cc:1685] finished
0709 12:16:08.001503 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:08.001713 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104162,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":25217792,"thread_start_us":277832,"threads_started":2,"wal-append.queue_time_us":185221}
I20260709 12:16:08.034226 23320 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:08.035189 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:08.039361 23469 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:08.036119 30462 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.041613 23463 raft_consensus.cc:2393] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:08.022348 30272 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.042258 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.045928 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:08.049463 30403 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:08.049940 30403 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.033197 30417 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.038965 30452 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:08.051345 30452 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.050968 30403 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.051843 30452 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:08.052474 30403 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.041131 30462 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:08.066113 30466 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.133s	user 0.000s	sys 0.002s
I20260709 12:16:08.029860 30317 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.075228 30455 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:08.075628 30317 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.082695 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:08.036581 30226 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:16:08.081897 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.086302 30465 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:16:08.084267 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 43822.1us
I20260709 12:16:08.089556 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:08.040934 30409 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.095984 30435 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:08.096470 30435 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:08.096843 30435 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:08.091405 30465 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.102667 30465 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.102928 30297 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.078882 30102 consensus_queue.cc:237] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.106698 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.107451 23319 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:08.113132 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:08.114545 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.090221 30429 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:16:08.125552 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.122897 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:08.128553 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.104969 30401 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.094871 30448 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.114421 30341 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:08.149729 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.125708 30417 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:08.161674 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.118199 30257 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:16:08.172739 30452 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.172739 30272 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.129630 30456 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:16:08.197239 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.200372 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.201184 23317 raft_consensus.cc:3060] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.130038 30436 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:16:08.195040 30297 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.192473 30417 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:08.198678 30356 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:08.173012 30409 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.219087 30462 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:08.250360 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.232285 30417 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:16:08.273168 23434 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.723s (SENT)
W20260709 12:16:08.307123 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 126192us
I20260709 12:16:08.308569 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:08.287256 23980 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.311134 23464 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:16:08.310463 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.314541 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.315164 23469 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:08.311308 30168 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.315869 30168 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:08.316450 30462 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.260051 30448 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.319571 30172 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:08.321203 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.323913 30179 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.326707 24013 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.328627 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.330142 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.330802 24009 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:08.330981 30452 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.340026 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.308738 30341 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.340026 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.349030 23316 raft_consensus.cc:3060] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:16:08.315079 30272 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:08.381167 30479 log.cc:927] Time spent T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:16:08.347043 30463 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.386250 30409 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.378720 30435 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:08.370932 30341 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:08.388928 30435 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.389935 24010 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.390329 30341 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.394438 30463 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.398294 30179 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:08.399087 23319 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:16:08.372274 30436 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:16:08.413385 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.434985 23980 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:08.441421 23317 raft_consensus.cc:2468] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:08.426573 30429 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.452524 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.454221 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.454825 23315 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:08.459117 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 185994us
I20260709 12:16:08.460095 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.460191 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.465610 24016 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:08.466118 24009 raft_consensus.cc:1240] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:08.470343 30451 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:08.471103 30451 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.450139 30465 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.475656 30465 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.457299 30429 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.456322 30172 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:08.477885 30172 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.479463 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:08.480207 23389 raft_consensus.cc:2393] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 22: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:08.482085 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:08.482661 23389 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:08.483086 30429 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.486291 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.453841 30428 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:16:08.492548 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.503546 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:08.509395 30469 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.583s	user 0.018s	sys 0.025s
I20260709 12:16:08.511479 23391 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:16:08.512742 23394 raft_consensus.cc:1240] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:16:08.480247 30172 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.537593 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.558970 23466 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.535977 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.560099 23468 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 17: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:08.516968 23362 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:08.537763 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:08.538460 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44149.3us
I20260709 12:16:08.541213 30401 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.541165 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.514556 30428 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.542892 30456 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:08.543095 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.544133 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.520939 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.522295 23464 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 21. Current term is 22. Ops: []
I20260709 12:16:08.565863 23469 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:08.539920 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.531177 23387 raft_consensus.cc:2468] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:08.550146 30429 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.565047 23470 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:08.537853 30451 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:08.523408 30257 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.528532 30226 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.513458 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.548915 30417 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:08.557030 23315 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:08.524575 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.523602 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.560848 23462 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 8 FOLLOWER]: Advancing to term 10
W20260709 12:16:08.561420 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 44551.4us
I20260709 12:16:08.562407 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.577446 23465 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:08.537312 30358 raft_consensus.cc:3055] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:08.568032 24016 raft_consensus.cc:2368] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17.
I20260709 12:16:08.578132 30358 raft_consensus.cc:740] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.578342 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.579120 30358 consensus_queue.cc:260] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.579733 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.580260 30358 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.580518 24007 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.581959 23467 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:16:08.580542 30168 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:08.589011 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.589820 23470 raft_consensus.cc:2368] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17.
I20260709 12:16:08.590529 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.570307 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.591430 30168 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:08.591840 30168 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.592155 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.569109 30429 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.593590 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.594220 24009 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.595149 30429 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:08.574335 30257 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.578750 30428 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.598218 30257 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:08.598693 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:08.564558 30456 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:08.576117 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.601447 30456 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.601975 23463 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:08.602661 30456 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.557371 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:08.574229 24010 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.603972 24012 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:16:08.572273 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.590593 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:08.592181 30168 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.608438 23468 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:08.567260 23316 raft_consensus.cc:2468] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:08.574949 23982 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.611714 23469 raft_consensus.cc:2468] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:08.614864 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.564877 30341 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:08.616757 23462 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 10.
I20260709 12:16:08.619200 23465 raft_consensus.cc:1240] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:08.620393 23465 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:08.622017 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.622669 23461 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 14. Ops: []
W20260709 12:16:08.621320 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 46458.6us
I20260709 12:16:08.623895 23467 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:08.624766 23469 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:08.625906 23462 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:08.627216 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.627823 23321 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 14.
I20260709 12:16:08.620685 23980 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:08.622372 24011 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:08.632231 30299 raft_consensus.cc:2804] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:08.632853 30299 raft_consensus.cc:697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.634002 30299 consensus_queue.cc:237] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.605994 30226 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.639081 30226 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:08.641230 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 190717us
I20260709 12:16:08.645016 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.646027 24010 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:08.647266 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.647933 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:08.649344 23318 raft_consensus.cc:2368] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 9. Current term is 10.
I20260709 12:16:08.651003 30263 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:08.651420 30263 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.651783 30263 raft_consensus.cc:3060] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.625870 30465 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:08.616389 30400 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:16:08.663055 23360 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:08.667594 30485 raft_consensus.cc:2804] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:08.670051 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:08.689143 23321 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:08.690114 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.662411 30465 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.691601 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.678329 30485 raft_consensus.cc:697] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:08.696713 23360 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:08.703240 23462 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:16:08.705422 23436 leader_election.cc:400] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:16:08.705776 23436 leader_election.cc:403] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:16:08.705621 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.705935 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:08.688118 30429 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:08.712131 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.712884 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.713272 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.714090 23321 raft_consensus.cc:2368] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 11. Current term is 12.
I20260709 12:16:08.712177 30481 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:08.714663 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:08.713510 23312 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 14.
I20260709 12:16:08.715485 23392 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:08.716526 23391 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:08.717574 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.723393 30417 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:08.696976 30465 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.733757 23312 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:08.735256 30462 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:08.713209 30429 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.737731 30429 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:08.743054 30485 consensus_queue.cc:237] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:08.746825 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 47080.7us
W20260709 12:16:08.749771 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 48970.5us
I20260709 12:16:08.752058 23466 raft_consensus.cc:2468] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:08.751966 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:08.762099 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.765067 23468 raft_consensus.cc:2468] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:16:08.772758 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.775080 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.771153 30436 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:08.779452 30436 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:08.779836 30436 raft_consensus.cc:3060] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.782910 30330 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:08.782943 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.752642 30481 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:08.785668 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.789770 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.790006 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.790992 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.791970 30482 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:08.792498 30482 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:08.793676 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47110.9us
I20260709 12:16:08.794672 30482 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:08.795279 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.034s (SENT)
I20260709 12:16:08.795352 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.795859 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.795985 30481 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:08.797096 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.073s (SENT)
W20260709 12:16:08.791025 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70141us
I20260709 12:16:08.798745 30418 consensus_queue.cc:1059] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:08.712026 30463 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 12:16:08.799440 30260 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:08.801330 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.802073 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:08.802107 23434 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:08.796618 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.777107 23982 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:08.803550 23314 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.770984 30428 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:08.810474 23982 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:08.810729 30428 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.811143 30330 raft_consensus.cc:2804] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:08.811861 30035 raft_consensus.cc:2804] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:08.811905 30330 raft_consensus.cc:697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.812582 30035 raft_consensus.cc:697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.759555 30172 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:16:08.809127 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:08.813536 30035 consensus_queue.cc:237] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.809402 23315 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:08.811246 30428 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:08.810675 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.761891 30417 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:08.824214 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:08.832052 23362 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.804739 30419 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:16:08.807547 30260 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.829465 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:08.841679 23391 raft_consensus.cc:606] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:08.843253 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.843616 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.844192 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.838264 30302 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:08.830392 30172 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:16:08.853299 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:08.857617 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60545.7us
I20260709 12:16:08.857999 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:08.858300 23434 leader_election.cc:336] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.085s (SENT)
I20260709 12:16:08.858712 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.853765 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:08.858723 23462 raft_consensus.cc:2393] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:08.876538 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:08.860594 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.841162 30260 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:08.851478 30302 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.860710 23982 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:08.881459 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.885102 30317 raft_consensus.cc:2804] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:08.885802 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:08.885828 30317 raft_consensus.cc:697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:08.887138 30317 consensus_queue.cc:237] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.830229 30417 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:08.845415 30438 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:08.859944 30484 raft_consensus.cc:2804] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:08.813246 30330 consensus_queue.cc:237] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:08.922405 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64130.9us
W20260709 12:16:08.923270 23434 leader_election.cc:336] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.116s (SENT)
I20260709 12:16:08.924341 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:08.903820 30492 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:16:08.891160 30302 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:08.891153 30272 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:16:08.932809 30276 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:08.931568 30484 raft_consensus.cc:697] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:08.904217 30438 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.960718 30438 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:16:08.964532 23434 leader_election.cc:336] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.170s (SENT)
I20260709 12:16:08.904763 30168 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.013480 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.024825 23463 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:08.977262 23390 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 18. Ops: []
I20260709 12:16:09.021606 23464 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:08.973289 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:08.979728 30400 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:16:08.989873 30493 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:16:09.008461 30417 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.053153 23396 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 19. Ops: []
I20260709 12:16:08.973809 30452 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:09.052301 23395 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:09.063917 23315 raft_consensus.cc:3055] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:09.067889 23315 raft_consensus.cc:740] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:09.078147 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.051924 30400 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:09.084054 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:09.084785 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 121379us
I20260709 12:16:09.085372 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:16:09.085492 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 2.248s (SENT)
I20260709 12:16:09.086184 23391 raft_consensus.cc:2393] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:09.005364 30429 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.088862 30429 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:09.088764 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.090793 23391 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 18.
I20260709 12:16:09.092496 24014 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:09.086014 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.095670 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:08.976521 30263 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.063632 30272 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:16:09.109100 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 271219us
I20260709 12:16:09.110816 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.049157 30102 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:16:09.045284 30436 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.082113 30491 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:09.107757 23315 consensus_queue.cc:260] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.123626 23387 raft_consensus.cc:2368] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:16:09.066430 23215 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 10 to 20. New cstate: current_term: 20 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:09.121143 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.108851 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.125128 23395 raft_consensus.cc:3060] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:09.128543 23396 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:09.067939 30465 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.066330 30299 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:16:09.132872 23464 raft_consensus.cc:1240] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 21. Current term is 22. Ops: []
I20260709 12:16:09.106750 30417 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:09.136526 30502 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:09.137686 23315 raft_consensus.cc:3060] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:09.152174 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 66741.2us
I20260709 12:16:09.128892 30491 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.157234 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:09.159991 30302 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.117687 30172 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:16:08.974346 30484 consensus_queue.cc:237] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.169571 23434 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.179606 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.168036 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:09.179181 30500 log.cc:927] Time spent T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:16:09.192072 23287 leader_election.cc:400] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17.
I20260709 12:16:09.193205 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.194790 23287 leader_election.cc:403] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:09.195464 23436 leader_election.cc:400] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 19. Current term is 20.
I20260709 12:16:09.198251 23434 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.202257 23395 raft_consensus.cc:2410] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:16:09.203235 30419 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:09.203343 30502 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17.
I20260709 12:16:09.210304 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.212044 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.212814 23395 raft_consensus.cc:2393] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:16:09.214748 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 46789.9us
I20260709 12:16:09.215513 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.216100 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:09.216914 23315 raft_consensus.cc:1275] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:16:09.217278 23315 pending_rounds.cc:85] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 13
I20260709 12:16:09.183987 30417 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:09.220600 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.221055 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:09.165261 30428 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.189297 30356 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:09.222927 30486 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:09.224488 30485 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:09.229764 30502 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:09.221313 30272 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:09.218482 30419 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:09.234133 30419 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.199150 30168 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.233203 30272 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:16:09.213508 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 136011us
I20260709 12:16:09.217542 23315 pending_rounds.cc:107] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:16:09.259497 23315 raft_consensus.cc:2889] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:09.234754 30419 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.290077 24010 raft_consensus.cc:2368] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:09.221832 30482 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:09.296670 23387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
W20260709 12:16:09.284444 30509 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:16:09.184399 30400 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:09.298458 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 77954us
I20260709 12:16:09.301590 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.297286 23387 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:16:09.314698 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 93728.3us
I20260709 12:16:09.316398 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:09.373620 23392 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: []
W20260709 12:16:09.424983 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 123545us
I20260709 12:16:09.428318 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:09.422807 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 192241us
I20260709 12:16:09.300001 30035 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.309525 30491 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:09.421057 30501 log.cc:927] Time spent T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:16:09.379194 30502 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:16:09.384994 30400 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:09.450182 23287 leader_election.cc:400] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17.
I20260709 12:16:09.430104 30516 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:09.456571 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.457363 30465 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:09.442109 30276 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:16:09.446499 30263 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.480458 30502 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:09.481683 23387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:09.482307 23387 raft_consensus.cc:606] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:09.495488 23389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1301) took 1010 ms. Trace:
I20260709 12:16:09.499495 23387 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:09.500166 23387 raft_consensus.cc:606] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.500509 23387 raft_consensus.cc:614] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:09.494415 30511 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:09.487470 30516 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.445576 30035 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.496011 23389 rpcz_store.cc:276] 0709 12:16:08.485559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:08.486588 (+  1029us) service_pool.cc:224] Handling call
0709 12:16:08.486927 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:08.821599 (+334672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:08.821633 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:08.830747 (+  9114us) log.cc:844] Serialized 52 byte log entry
0709 12:16:09.048448 (+217701us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:16:09.067780 (+ 19332us) log.cc:844] Serialized 14 byte log entry
0709 12:16:09.074613 (+  6833us) log.cc:844] Serialized 14 byte log entry
0709 12:16:09.081974 (+  7361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:09.082066 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:09.160083 (+ 78017us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b18004b15a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a65873f3 00007ff0a65897c1 00007ff0a6589779 00007ff0a6589709 00007ff0a65896d1 00007ff0a65889cd 0000000000388634 0000000000388479 00007ff0a656a83a
0709 12:16:09.495144 (+335061us) raft_consensus.cc:1685] finished
0709 12:16:09.495187 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:09.495410 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49656,"prepare.queue_time_us":77184,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":62221,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":5541,"spinlock_wait_cycles":7945856,"thread_start_us":512105,"threads_started":2,"wal-append.queue_time_us":280757}
I20260709 12:16:09.508942 30516 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.509835 30035 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.513226 30503 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:16:09.518690 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 52608.6us
I20260709 12:16:09.519454 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
W20260709 12:16:09.514885 30260 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.004s
I20260709 12:16:09.520853 23389 raft_consensus.cc:606] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:16:09.517329 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 66019.4us
I20260709 12:16:09.522203 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.501417 30511 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.530741 30260 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.518860 30503 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:09.536811 30260 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:09.537176 30438 consensus_meta.cc:287] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.004s
I20260709 12:16:09.540169 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:09.548062 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:09.557749 24009 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 19: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:09.565843 23388 raft_consensus.cc:1240] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 13. Current term is 15. Ops: []
W20260709 12:16:09.570555 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Timed out: RequestConsensusVote RPC to 127.21.150.193:46055 timed out after 1.555s (SENT)
I20260709 12:16:09.571237 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.572223 30302 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.573792 30436 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.574450 30436 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:09.579583 30511 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.541141 30438 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.584563 30438 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:09.588079 30514 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.136s	user 0.000s	sys 0.003s
I20260709 12:16:09.589092 23394 raft_consensus.cc:2368] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15.
I20260709 12:16:09.589915 30503 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.591230 30503 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:09.594419 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:09.595036 23468 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:09.565843 30521 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:09.596902 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.598440 30179 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:09.599309 30299 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:09.559087 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:09.602875 23469 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:09.597308 30521 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:09.609908 30428 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.610292 30521 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.611356 30521 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:09.613349 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.614074 23466 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:16:09.620615 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:09.622738 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:09.623703 23461 raft_consensus.cc:2393] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 20: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:09.626585 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.615118 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:09.612730 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:09.632085 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:09.633050 23468 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:09.634032 23470 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:16:09.635090 23468 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:09.635146 23465 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:16:09.640802 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.641464 23395 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:09.642563 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.643376 23394 raft_consensus.cc:2410] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 18 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:16:09.649407 23436 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.646967 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:09.651630 30448 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:09.653656 30463 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:09.654042 30448 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:09.654184 30463 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting leader election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:09.654506 30448 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:09.654683 30463 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:09.650552 30452 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:09.651764 30467 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:16:09.668301 30330 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:09.658480 30519 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:16:09.671085 30419 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:09.675616 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.682390 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:09.677075 30035 consensus_queue.cc:1048] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 12:16:09.685822 30510 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.241s	user 0.002s	sys 0.000s
I20260709 12:16:09.691919 23469 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:09.698056 24010 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:16:09.664238 24009 raft_consensus.cc:2468] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:09.718152 24014 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:16:09.724449 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 192667us
I20260709 12:16:09.725237 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.725926 23436 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.727545 23982 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:09.728566 30430 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:16:09.731554 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:09.734565 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:09.740403 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:09.737669 30419 raft_consensus.cc:2804] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:09.729257 30430 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:09.761914 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:09.719417 30467 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 12:16:09.769222 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120028us
I20260709 12:16:09.776894 23362 leader_election.cc:400] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:09.777318 23362 leader_election.cc:403] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:09.778486 23362 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:09.761098 23215 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 20, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 20 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:09.756639 30419 raft_consensus.cc:697] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:09.787389 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:09.790004 23320 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:09.788117 30532 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:09.792337 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 67189.8us
I20260709 12:16:09.781412 30482 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:16:09.794795 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:09.797526 23312 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:09.806066 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:09.807044 23470 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:09.807338 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:09.808079 24009 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:09.808647 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:09.809273 23469 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:09.791083 30257 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:16:09.792769 30503 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:16:09.835444 30524 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:09.836305 30523 consensus_queue.cc:1059] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:16:09.836179 30419 consensus_queue.cc:237] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.836833 23318 raft_consensus.cc:1275] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:09.845325 23469 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:09.850673 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.870613 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.828087 30276 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:09.827462 30438 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.879468 23315 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:16:09.871201 30504 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:09.882536 30494 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:09.882620 23362 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.885777 23320 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:09.834239 30430 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.832434 30401 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:16:09.827838 30400 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:16:09.901579 30448 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.910362 23464 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:16:09.912292 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 61713.8us
I20260709 12:16:09.922091 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.924592 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.925328 23392 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 19: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:09.922076 23387 raft_consensus.cc:1275] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 12:16:09.928707 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.930764 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:09.928042 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.934998 23392 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 20: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:09.918962 30502 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:09.938479 23394 raft_consensus.cc:1275] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 12:16:09.924538 30341 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:09.939728 30486 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:09.941768 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:09.928893 23387 pending_rounds.cc:85] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 15
I20260709 12:16:09.943396 23387 pending_rounds.cc:107] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:16:09.943611 23387 raft_consensus.cc:2889] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:09.940598 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:09.870802 30521 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:16:09.796945 30532 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.932785 23388 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:09.940310 30486 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.948392 23215 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:09.949549 23392 raft_consensus.cc:1275] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:09.955749 30330 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:09.961750 30276 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.964039 30448 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.967815 30463 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.971632 30486 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:09.973614 30526 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:16:09.973846 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:09.974486 23394 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:09.976248 30503 raft_consensus.cc:3055] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:16:09.976626 30503 raft_consensus.cc:740] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:09.977260 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:09.877660 30524 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.871333 30531 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:16:09.918735 30438 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.924069 30504 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:09.977421 30503 consensus_queue.cc:260] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.983807 30503 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:09.946516 30523 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:09.984562 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.985334 30523 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:09.977948 30521 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:16:09.985951 30523 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:09.977851 23317 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:09.988619 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 57938.4us
I20260709 12:16:09.978951 23313 raft_consensus.cc:1275] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:16:09.989785 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:09.994680 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:09.995572 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:09.996111 23468 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:09.999004 23467 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:16:09.998778 30451 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.016255 23317 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
W20260709 12:16:10.042425 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 52722.3us
I20260709 12:16:10.045321 23470 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
W20260709 12:16:10.040205 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 43972.5us
I20260709 12:16:09.981423 30533 raft_consensus.cc:3055] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:09.981047 30532 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:09.979733 30524 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.058005 23287 leader_election.cc:400] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15.
I20260709 12:16:10.058364 23287 leader_election.cc:403] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:16:09.984499 30504 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:10.066874 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:10.073889 23215 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:10.041186 30523 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.098338 30486 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:10.051841 30533 raft_consensus.cc:740] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:10.106662 23468 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:10.104921 23362 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:10.063396 30438 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.082988 30524 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:10.095945 30341 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.050719 30451 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:10.114099 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56232.9us
I20260709 12:16:10.139380 30462 consensus_queue.cc:1048] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:10.140175 30520 consensus_queue.cc:1048] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:10.140862 30465 consensus_queue.cc:1048] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:10.146699 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:10.147596 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:10.110455 30486 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.129302 30531 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15.
I20260709 12:16:10.111452 30463 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.155408 30302 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:10.183775 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:10.103781 30502 consensus_queue.cc:1059] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:16:10.186496 23362 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:10.168192 30524 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:10.190647 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:10.193822 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:10.201031 23468 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:10.207252 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:10.207425 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:10.208029 23320 raft_consensus.cc:2393] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:10.208118 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:10.209012 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:10.209502 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:10.226799 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:10.226199 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:10.211555 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:10.211047 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:10.188073 30486 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:10.230154 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:10.231925 23313 raft_consensus.cc:2393] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 15: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:10.233574 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:10.235998 23388 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:10.239132 23312 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 19.
I20260709 12:16:10.165529 30341 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.205173 30428 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:10.245201 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:10.251278 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:10.225641 30168 raft_consensus.cc:2804] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:10.241966 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:10.188596 30400 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:16:10.255919 23389 raft_consensus.cc:2393] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 20: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:10.256037 23470 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:10.258049 23470 raft_consensus.cc:1240] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:10.229096 30430 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:10.258352 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:10.209911 30532 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:16:10.232015 30302 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:10.261889 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:10.239440 30419 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:16:10.243008 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:10.239224 30543 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:10.300478 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 59607.2us
I20260709 12:16:10.260572 30456 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:10.254974 30539 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:10.260288 30523 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:10.268649 30276 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:10.268967 30168 raft_consensus.cc:697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:10.249775 30531 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:10.287277 30432 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:10.287760 30532 consensus_queue.cc:1243] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer 6253b834ffb744579b46fcede2fc0e91 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:16:10.342489 30168 consensus_queue.cc:237] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:10.345810 30544 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:16:10.342242 30302 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.350576 23312 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:16:10.297122 30430 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:10.365681 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:10.354979 23215 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index 5 to 11, term changed from 15 to 20, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to <none>, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 20 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:10.340605 30432 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:10.381073 23388 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:10.365974 30539 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.350538 30523 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.311136 30543 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.377543 30432 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:10.361588 30531 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:10.402154 30482 consensus_meta.cc:287] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.001s
I20260709 12:16:10.302081 30524 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.366189 30555 consensus_queue.cc:1059] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:16:10.397526 30539 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:10.417197 30542 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:16:10.389348 30428 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:10.470237 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:10.473620 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:10.474869 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 110396us
I20260709 12:16:10.420887 30482 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405) with higher term. Message: Invalid argument: T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20.
I20260709 12:16:10.480189 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:10.506063 30541 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:16:10.471839 30463 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:10.511029 30540 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
I20260709 12:16:10.512509 30555 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:10.565562 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:10.460733 30430 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.556620 30428 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.529302 30438 consensus_queue.cc:1048] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:16:10.553941 30568 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:10.572546 23395 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
W20260709 12:16:10.555455 30549 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 12:16:10.569824 30482 raft_consensus.cc:993] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:16:10.569957 30547 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:16:10.520594 30463 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:10.589567 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 60114.3us
I20260709 12:16:10.597851 30401 raft_consensus.cc:993] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:10.598410 30401 raft_consensus.cc:1081] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:10.599938 30451 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:10.577517 30565 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:16:10.614727 30546 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:16:10.593261 30257 raft_consensus.cc:1081] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:16:10.639426 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 129599us
I20260709 12:16:10.568930 30533 consensus_queue.cc:260] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.578166 30555 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.656648 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:10.653010 23360 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:10.660224 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:10.663669 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:10.664283 30550 consensus_queue.cc:1048] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:10.664745 30569 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:10.618392 30486 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.677606 30257 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:16:10.665306 30569 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:16:10.718278 30558 log.cc:927] Time spent T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:16:10.724511 23394 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 19. Ops: []
I20260709 12:16:10.668826 30543 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.646461 30565 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:10.727103 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 61819.1us
W20260709 12:16:10.721895 30549 log.cc:927] Time spent T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:16:10.687260 30531 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.740080 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:10.747418 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 152775us
W20260709 12:16:10.744055 30560 log.cc:927] Time spent T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.160s	user 0.000s	sys 0.004s
W20260709 12:16:10.785104 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 107025us
I20260709 12:16:10.788319 30570 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:16:10.754529 30257 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
W20260709 12:16:10.794493 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48329.5us
I20260709 12:16:10.781334 30465 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:10.825007 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:10.783033 30432 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.795783 30451 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 12:16:10.809798 30503 consensus_meta.cc:287] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.826s	user 0.014s	sys 0.000s
I20260709 12:16:10.851708 23314 raft_consensus.cc:3060] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:10.855108 30432 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.858691 30555 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:10.798584 30257 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.795215 30569 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:10.863623 30257 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:10.871065 23315 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:10.871901 23315 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:10.872363 23315 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:10.869405 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:10.876480 23316 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:10.876971 23316 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:10.904115 23392 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:16:10.866438 30226 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:10.907800 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:10.916194 23392 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:10.932014 23315 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.927318 23316 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.943122 23315 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.948776 23314 raft_consensus.cc:2468] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:16:10.923859 30465 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:16:10.952028 30581 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:16:10.955287 23392 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 23.
I20260709 12:16:10.965827 23316 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.953835 30586 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.939184 30486 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:10.915977 30419 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 12:16:10.973773 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 191396us
I20260709 12:16:10.973194 30452 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:10.985152 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:10.973086 30586 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:10.999125 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:10.972200 30565 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:10.992565 30586 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.021160 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:11.022147 24009 raft_consensus.cc:1275] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:16:10.996949 30452 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.028690 30452 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:11.031373 30587 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:16:11.037976 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:11.040774 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:10.994808 30465 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.053282 30517 raft_consensus.cc:993] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:11.042429 30419 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:11.035084 30452 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.068666 30577 raft_consensus.cc:1081] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:16:11.070514 30562 log.cc:927] Time spent T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.500s	user 0.000s	sys 0.002s
I20260709 12:16:11.080200 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.081110 23469 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
I20260709 12:16:11.084195 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.096179 23320 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:11.098592 23320 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:11.113716 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:11.116878 23469 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:16:11.114740 23468 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:11.126631 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.128399 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:11.129060 23320 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.129602 23320 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:11.126706 23462 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:11.064376 30486 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.137178 23468 raft_consensus.cc:2468] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:11.091754 30448 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.099718 30419 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:11.159632 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.161389 23468 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:16:11.163707 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:11.168215 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:11.092562 30452 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.179332 23320 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.169951 30574 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:11.194741 30452 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:11.200124 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.209904 30574 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.203404 30609 consensus_queue.cc:1048] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:16:11.184401 30299 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.215438 30299 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.215984 30539 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.217985 30419 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:11.219772 30566 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.655s	user 0.003s	sys 0.000s
I20260709 12:16:11.172191 30482 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:11.223959 23314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42788 (request call id 3249) took 1060 ms. Trace:
I20260709 12:16:11.224510 23314 rpcz_store.cc:276] 0709 12:16:10.163243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.163561 (+   318us) service_pool.cc:224] Handling call
0709 12:16:11.223715 (+1060154us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54003c0d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:11.223874 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490337152}
I20260709 12:16:11.225035 23320 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.226593 23315 raft_consensus.cc:1275] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:16:11.157086 30486 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.206290 23215 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.210536 30574 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:11.205639 30521 raft_consensus.cc:3055] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:16:11.236393 30559 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.663s	user 0.002s	sys 0.000s
I20260709 12:16:11.238816 30317 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.239324 24014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1552) took 1002 ms. Trace:
I20260709 12:16:11.239466 30523 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.162060 30448 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.252396 23320 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 18. Current term is 19. Ops: []
I20260709 12:16:11.252238 23215 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:11.242122 30609 raft_consensus.cc:493] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.263139 24014 rpcz_store.cc:276] 0709 12:16:10.237350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.245744 (+  8394us) service_pool.cc:224] Handling call
0709 12:16:10.246046 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:10.372463 (+126417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.372500 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:10.372925 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:16:10.471644 (+ 98719us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:10.471693 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:10.473068 (+  1375us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:11.219023 (+745955us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:11.232298 (+ 13275us) raft_consensus.cc:1685] finished
0709 12:16:11.232342 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:11.239260 (+  6918us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3576,"raft.queue_time_us":45342,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3438,"spinlock_wait_cycles":17675648,"thread_start_us":220304,"threads_started":2,"wal-append.queue_time_us":187428}
W20260709 12:16:11.270385 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 96935.9us
I20260709 12:16:11.272891 23215 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
W20260709 12:16:11.278296 23217 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 222) took 37532 ms (client timeout 15000 ms). Trace:
W20260709 12:16:11.278725 23217 rpcz_store.cc:269] 0709 12:15:33.745991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:33.746223 (+   232us) service_pool.cc:224] Handling call
0709 12:16:11.278113 (+37531890us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:05.470377 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:05.623744 (+153367us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:05.623802 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:16:05.623841 (+    39us) tablet.cc:662] Decoding operations
0709 12:16:05.625100 (+  1259us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:05.625207 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:05.625236 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:16:05.626539 (+  1303us) tablet.cc:701] Row locks acquired
0709 12:16:05.628436 (+  1897us) write_op.cc:260] PREPARE: finished
0709 12:16:05.629172 (+   736us) write_op.cc:270] Start()
0709 12:16:05.629354 (+   182us) write_op.cc:276] Timestamp: P: 1783599365628590 usec, L: 0
0709 12:16:05.629402 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:16:05.810600 (+181198us) log.cc:844] Serialized 4014 byte log entry
0709 12:16:06.729607 (+919007us) op_driver.cc:464] REPLICATION: finished
0709 12:16:06.843485 (+113878us) write_op.cc:301] APPLY: starting
0709 12:16:06.843597 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:07.399186 (+555589us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:07.420955 (+ 21769us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:07.902968 (+482013us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:07.905985 (+  3017us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:16:07.906209 (+   224us) write_op.cc:312] APPLY: finished
0709 12:16:07.912018 (+  5809us) log.cc:844] Serialized 97 byte log entry
0709 12:16:08.039215 (+127197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:08.046258 (+  7043us) write_op.cc:454] Released schema lock
0709 12:16:08.083747 (+ 37489us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:08.158541 (+ 74794us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:08.190487 (+ 31946us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21380,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":113559,"apply.run_cpu_time_us":42046,"apply.run_wall_time_us":1284089,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45967,"num_ops":10,"prepare.queue_time_us":75470,"prepare.run_cpu_time_us":35769,"prepare.run_wall_time_us":541760,"raft.queue_time_us":236343,"raft.run_cpu_time_us":23449,"raft.run_wall_time_us":79293,"replication_time_us":1100007,"thread_start_us":547310,"threads_started":5,"wal-append.queue_time_us":652945,"wal-append.run_cpu_time_us":50090,"wal-append.run_wall_time_us":2397279}]]}
I20260709 12:16:11.250535 30521 raft_consensus.cc:740] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:11.291222 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.276639 30341 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.284457 30448 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.305258 30341 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:11.308177 30455 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:11.309005 23362 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:11.302171 30539 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.319460 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:11.290889 30523 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.323362 23215 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 10 to 12, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:11.330402 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.331240 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.331501 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.320276 23391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1333) took 1003 ms. Trace:
I20260709 12:16:11.336550 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.269127 30317 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.338011 23320 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:16:11.340179 23469 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 19. Ops: []
I20260709 12:16:11.340631 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.343062 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:11.343382 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.344215 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.344437 30506 raft_consensus.cc:2804] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:11.345024 30506 raft_consensus.cc:697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:11.345238 30418 raft_consensus.cc:993] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:11.345950 30418 raft_consensus.cc:1081] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:11.346004 30506 consensus_queue.cc:237] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.348822 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.289886 30609 raft_consensus.cc:515] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.351145 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.352020 23320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:11.356492 23320 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:11.356567 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:11.356940 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:11.352581 23469 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:11.358150 23467 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:16:11.338264 23215 catalog_manager.cc:5697] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.356863 23320 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:11.353229 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.369711 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:11.369697 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:11.371023 23393 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:11.371225 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:11.371040 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.373605 23393 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:11.409027 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.334920 23391 rpcz_store.cc:276] 0709 12:16:10.316542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.316680 (+   138us) service_pool.cc:224] Handling call
0709 12:16:10.317009 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:10.353631 (+ 36622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.353663 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:10.354120 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:16:10.467083 (+112963us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:16:10.533742 (+ 66659us) log.cc:844] Serialized 14 byte log entry
0709 12:16:10.646238 (+112496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:10.646372 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:11.235785 (+589413us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:11.315680 (+ 79895us) raft_consensus.cc:1685] finished
0709 12:16:11.320074 (+  4394us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:11.320216 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42301,"prepare.queue_time_us":103355,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":47968,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":4798,"spinlock_wait_cycles":3716864,"thread_start_us":211549,"threads_started":3,"wal-append.queue_time_us":161713}
I20260709 12:16:11.406853 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.414160 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:11.414820 23387 raft_consensus.cc:2368] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 22. Current term is 23.
I20260709 12:16:11.416476 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:11.417071 23396 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:11.417362 23396 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
W20260709 12:16:11.421540 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 49288.2us
I20260709 12:16:11.422447 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:11.422407 23215 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.425283 23467 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:11.425920 24014 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
I20260709 12:16:11.439831 23320 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.411955 23393 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.447752 23393 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:11.350160 30341 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:11.441608 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:11.349578 30543 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.442015 23215 catalog_manager.cc:5697] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.446143 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:11.455699 23387 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:16:11.456115 23387 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:11.454854 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:11.454198 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:11.458277 23470 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:11.407130 30555 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.453586 23396 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.465178 23396 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:11.465938 23317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 3254) took 1209 ms. Trace:
I20260709 12:16:11.325922 30539 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.429836 30615 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.466404 23317 rpcz_store.cc:276] 0709 12:16:10.256572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.256761 (+   189us) service_pool.cc:224] Handling call
0709 12:16:10.257038 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:10.459540 (+202502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.459575 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:10.459626 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:16:10.459655 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:11.465696 (+1006041us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54002756d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:11.465739 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:11.465874 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465116032}
I20260709 12:16:11.490260 23320 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.317529 30521 consensus_queue.cc:260] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.421486 30565 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.463227 23215 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 17, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 17 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.370039 30523 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:11.460120 30559 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:16:11.500483 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 78133.7us
I20260709 12:16:11.484225 23469 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:16:11.456542 30341 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:11.457024 30504 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:11.507895 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:11.499105 23387 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:11.506243 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 220800us
I20260709 12:16:11.476006 30299 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:11.473672 30438 consensus_queue.cc:1175] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 18 status { last_received { term: 18 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 18 index: 13 } }
I20260709 12:16:11.535065 30620 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.535617 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
W20260709 12:16:11.536175 30614 log.cc:927] Time spent T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:16:11.554809 23470 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:11.538115 30622 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:11.543764 23360 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:11.543949 30605 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:16:11.533015 23215 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 19, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 19 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.510190 30341 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:11.462692 30543 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.551698 24012 raft_consensus.cc:606] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:11.520385 30419 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.570416 30419 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:11.571465 30612 log.cc:927] Time spent T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:16:11.574704 30419 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.575044 30463 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:11.576177 30622 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.576758 30622 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.578233 30341 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.579113 30622 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.579704 30622 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:11.580281 30341 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.581687 23389 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: []
W20260709 12:16:11.567859 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 74651.3us
I20260709 12:16:11.497675 30555 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.505666 30565 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.585718 23393 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:16:11.586499 23215 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 15 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 20 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.502625 30615 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:11.589155 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 235778us
I20260709 12:16:11.602351 23215 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.617237 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:11.618156 23468 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:11.596153 30622 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.621819 23390 raft_consensus.cc:1240] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 19. Ops: []
I20260709 12:16:11.621622 30622 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.624796 30622 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:11.625562 30622 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.582242 30486 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:11.628901 23215 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.635471 23394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 3115) took 1443 ms. Trace:
I20260709 12:16:11.635820 23394 rpcz_store.cc:276] 0709 12:16:10.192108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.192260 (+   152us) service_pool.cc:224] Handling call
0709 12:16:10.192544 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:10.726763 (+534219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:11.635273 (+908510us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54002e9d54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:11.635426 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223014272}
I20260709 12:16:11.638828 23215 catalog_manager.cc:5697] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 17, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:11.659021 30616 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:16:11.659905 23215 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:11.683393 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:11.688700 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:11.691946 23468 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:11.693761 23215 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 17, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 17 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.690706 30455 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.702279 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:11.569314 30620 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.703523 23394 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:11.629516 30575 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:16:11.734884 23396 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:16:11.724498 23215 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 10 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 15 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:11.505625 30276 consensus_queue.cc:1175] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "da1e2fd157894634bf92d6c82c06ec64" responder_term: 19 status { last_received { term: 17 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 17. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:16:11.705611 30317 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:11.754318 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 246503us
W20260709 12:16:11.756577 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 54136.3us
I20260709 12:16:11.765635 23982 leader_election.cc:400] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 22. Current term is 23.
I20260709 12:16:11.765995 23982 leader_election.cc:403] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:11.702694 30455 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.758119 30609 leader_election.cc:290] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.783624 30299 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 23
W20260709 12:16:11.785369 30579 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.992s	user 0.000s	sys 0.010s
I20260709 12:16:11.796180 30629 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.836684 23318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 2045) took 1039 ms. Trace:
I20260709 12:16:11.810667 30615 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:11.811828 30555 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:11.812784 30617 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:16:11.770015 30539 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.866551 30543 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.836993 30523 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.845249 23215 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 8 to 10, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 10 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:11.763408 30432 consensus_queue.cc:1048] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:16:11.867535 30539 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:11.883085 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:11.626226 30622 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:11.885775 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 150571us
I20260709 12:16:11.891031 23215 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 9 to 14, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.848268 30629 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.890563 30539 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.907132 30299 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721) with higher term. Message: Invalid argument: T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 for earlier term 22. Current term is 23.
I20260709 12:16:11.837798 23318 rpcz_store.cc:276] 0709 12:16:10.796807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.797053 (+   246us) service_pool.cc:224] Handling call
0709 12:16:10.801834 (+  4781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:10.811893 (+ 10059us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.811927 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:11.344125 (+532198us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b48000d08d8. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0ae59daa1 00007ff0ae594b37 00007ff0ae5949ca 00007ff0ae68c26f 00007ff0ae68bf37 00007ff0ae6a055c 00007ff0ae6a0508 00007ff0ae6a0488 00007ff0ae6a0443 00007ff0ae69f6ec 00007ff0ad5c9a96
0709 12:16:11.348569 (+  4444us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b2c00028668. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7d372 00007ff0a96afa01 00007ff0a967443a 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2
0709 12:16:11.349082 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:16:11.632186 (+283104us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:16:11.719782 (+ 87596us) log.cc:844] Serialized 14 byte log entry
0709 12:16:11.835356 (+115574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:11.835463 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:11.835820 (+   357us) raft_consensus.cc:1685] finished
0709 12:16:11.835843 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:11.836542 (+   699us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9532,"prepare.queue_time_us":58113,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":4637,"raft.run_cpu_time_us":3646,"raft.run_wall_time_us":536389,"spinlock_wait_cycles":117960064,"thread_start_us":397646,"threads_started":2,"wal-append.queue_time_us":387866}
I20260709 12:16:11.886128 30565 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:11.913486 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 220662us
I20260709 12:16:11.910684 30571 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:11.913218 23215 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 18, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 18 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:11.914930 23285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:11.919937 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:11.925411 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:11.928380 23215 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:11.930876 23215 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:11.932215 23215 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 11 to 15, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.933916 23215 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 12 to 19, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 19 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.922724 30620 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:11.943323 23215 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.936194 30633 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:11.953627 30633 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.966243 23388 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:11.967331 30637 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:11.967948 30637 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:11.918423 30571 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:11.970752 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:11.972687 24012 raft_consensus.cc:606] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
W20260709 12:16:11.981935 23395 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34554 (request call id 1318) took 2044 ms (client timeout 1690 ms). Trace:
W20260709 12:16:11.982414 23395 rpcz_store.cc:269] 0709 12:16:09.920969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:09.921373 (+   404us) service_pool.cc:224] Handling call
0709 12:16:11.965622 (+2044249us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54002e8e54. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96782fa 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c 00007ff0a7bb37c2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:11.965753 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282210688}
W20260709 12:16:11.990172 30631 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:16:11.993090 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:11.993985 24013 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:11.994980 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:12.003296 30636 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:16:12.003894 30637 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.004822 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.010082 30633 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.011418 30629 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.013330 30539 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.019577 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.021272 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.022035 24007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b055a413d01c460895f3bc5034afc461"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:12.022642 30622 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.024425 30571 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.024832 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.022667 24007 raft_consensus.cc:606] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:12.026253 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.026166 24007 raft_consensus.cc:614] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:12.028559 24010 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:16:12.028654 30533 raft_consensus.cc:3060] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:12.026889 30574 consensus_meta.cc:287] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.000s
I20260709 12:16:12.028024 23316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 3267) took 1159 ms. Trace:
I20260709 12:16:12.031759 30574 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.032572 23316 rpcz_store.cc:276] 0709 12:16:10.868243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.996678 (+128435us) service_pool.cc:224] Handling call
0709 12:16:10.996947 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.042399 (+ 45452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:11.042432 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:11.042478 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:11.042513 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.027774 (+985261us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54003f98d4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:12.027823 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.027960 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104042240}
W20260709 12:16:12.042959 23287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.035385 30574 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.033294 23394 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:12.056090 24013 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:12.060963 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:12.054149 30571 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.045462 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:12.045872 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.045497 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:12.064029 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.064179 23316 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:12.073557 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:12.076321 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:12.077332 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.079279 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.057471 30622 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.089509 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:12.089655 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 150431us
I20260709 12:16:12.091971 23362 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:12.092550 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:12.092882 23392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52270 (request call id 2510) took 1057 ms. Trace:
I20260709 12:16:12.093151 23389 raft_consensus.cc:606] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:12.094884 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:12.095456 23389 raft_consensus.cc:3055] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:12.095737 23389 raft_consensus.cc:740] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:12.095047 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:12.098471 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.099182 23395 raft_consensus.cc:2393] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 19: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:12.099890 24011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53950 (request call id 1558) took 1008 ms. Trace:
I20260709 12:16:12.079782 30503 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:12.077116 30452 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:16:12.057909 30550 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:16:12.077101 30532 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:12.099203 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:12.102916 23321 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 3266) took 1235 ms. Trace:
I20260709 12:16:12.100211 24011 rpcz_store.cc:276] 0709 12:16:11.091772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.092130 (+   358us) service_pool.cc:224] Handling call
0709 12:16:11.127443 (+ 35313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.284923 (+157480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:11.284972 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:16:11.285034 (+    62us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:16:11.285084 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.099656 (+814572us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54007f0154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:12.099700 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.099836 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361494912}
I20260709 12:16:12.134616 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:12.093245 23392 rpcz_store.cc:276] 0709 12:16:11.035946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.036105 (+   159us) service_pool.cc:224] Handling call
0709 12:16:11.036442 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:16:11.037886 (+  1444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:11.037916 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:16:11.042171 (+  4255us) log.cc:844] Serialized 52 byte log entry
0709 12:16:11.158156 (+115985us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:16:11.178408 (+ 20252us) log.cc:844] Serialized 14 byte log entry
0709 12:16:11.311121 (+132713us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18004b9e20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6584455 00007ff0a966c060 00007ff0a967e716 00007ff0a968f287 00007ff0a968f208
0709 12:16:11.543192 (+232071us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b4400022b4c. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a64adc85 00007ff0a64ae6a2 00007ff0a92201db 00007ff0a921baaf 00007ff0a920f86e 00007ff0a920ebb6 00007ff0a920e7d8 00007ff0a9218221 00007ff0a92181d9 00007ff0a9218169
0709 12:16:12.092493 (+549301us) spinlock_profiling.cc:243] Waited 88.3 ms on lock 0x7b540032f3a0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ef733 00007ff0a9674dbf 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c
0709 12:16:12.092545 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.092631 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:12.092676 (+    45us) raft_consensus.cc:1685] finished
0709 12:16:12.092690 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.092829 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1570,"prepare.queue_time_us":19465,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":70,"raft.queue_time_us":11651,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":8306,"spinlock_wait_cycles":211033344,"thread_start_us":393450,"threads_started":2,"wal-append.queue_time_us":225251}
I20260709 12:16:12.096323 23389 consensus_queue.cc:260] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.136613 23389 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:12.107010 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.138187 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4ebaab4fff4457a89f5a00d4f71ddc8" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.088828 30629 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:12.139732 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.140590 23396 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:16:12.068869 30571 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.142969 23313 raft_consensus.cc:1275] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:16:12.146051 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.146109 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.146706 23393 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:12.147765 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.148542 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.097160 30586 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:12.143462 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.079219 30539 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:16:12.155057 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.155697 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.104085 30620 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:12.089402 30455 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.106561 30452 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.168653 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.180552 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.165480 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.180867 23468 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:12.102134 30633 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:16:12.087037 30622 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.221719 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.222244 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.224059 30622 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.226536 23317 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
W20260709 12:16:12.217552 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56209.8us
I20260709 12:16:12.228267 23436 leader_election.cc:304] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.143420 30645 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:12.168356 23320 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:12.139288 30625 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:12.231776 23312 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:12.149957 30276 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:12.170939 30566 log.cc:927] Time spent T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:16:12.153862 30533 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:12.184305 30452 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:12.236037 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.237062 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:12.214761 30617 log.cc:927] Time spent T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:16:12.244160 30276 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.218894 30486 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.246407 30456 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:12.249303 30486 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.249847 30521 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:16:12.250952 30644 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:16:12.251991 30486 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.170962 30586 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.267148 30341 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.267760 30341 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.270280 30533 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:12.271319 30533 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.272066 23287 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:12.267151 23388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34554 (request call id 1337) took 1284 ms. Trace:
I20260709 12:16:12.276428 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:12.274289 23388 rpcz_store.cc:276] 0709 12:16:10.983044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.983800 (+   756us) service_pool.cc:224] Handling call
0709 12:16:11.965791 (+981991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.966764 (+   973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.266942 (+300178us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54002e8e50. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96738ad 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:12.267090 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2257676544}
I20260709 12:16:12.280820 30622 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:12.281883 30539 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:12.282642 30629 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:12.282893 30539 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:12.283317 30629 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:12.284235 30539 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.284332 30629 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.285538 30622 consensus_queue.cc:1059] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:12.286420 30620 raft_consensus.cc:3055] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:12.286787 30620 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:12.287420 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:12.288885 30622 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:12.289341 30622 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.289788 30622 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:12.291167 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.297662 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.299132 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.241319 30609 raft_consensus.cc:2749] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:12.301580 23465 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:12.303285 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.303887 23388 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
W20260709 12:16:12.326031 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 47734.2us
I20260709 12:16:12.331113 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.247498 30625 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:16:12.338845 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 93187.9us
I20260709 12:16:12.334292 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.316449 30633 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:12.300940 30645 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:12.325405 30503 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:12.340951 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.344254 23388 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:12.345057 30341 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:12.358088 30639 log.cc:927] Time spent T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:16:12.361649 30620 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.362897 30565 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.364513 30451 raft_consensus.cc:993] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.367776 23313 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42788 (request call id 3263) took 1876 ms. Trace:
I20260709 12:16:12.368345 23390 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52266 (request call id 3124) took 1016 ms. Trace:
I20260709 12:16:12.134174 23321 rpcz_store.cc:276] 0709 12:16:10.867920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.869942 (+  2022us) service_pool.cc:224] Handling call
0709 12:16:10.870362 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:10.885885 (+ 15523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.885931 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:10.907004 (+ 21073us) log.cc:844] Serialized 26 byte log entry
0709 12:16:10.981964 (+ 74960us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:16:11.065043 (+ 83079us) log.cc:844] Serialized 14 byte log entry
0709 12:16:12.086571 (+1021528us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b540019ce20. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95ef733 00007ff0a9674dbf 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c
0709 12:16:12.086629 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.086702 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:12.086758 (+    56us) raft_consensus.cc:1685] finished
0709 12:16:12.102716 (+ 15958us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.102858 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188326,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":31865,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":4403,"spinlock_wait_cycles":45856512,"thread_start_us":109002,"threads_started":2,"wal-append.queue_time_us":110162,"wal-append.run_cpu_time_us":3422,"wal-append.run_wall_time_us":1096199}
I20260709 12:16:12.393327 23313 rpcz_store.cc:276] 0709 12:16:10.490931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.491070 (+   139us) service_pool.cc:224] Handling call
0709 12:16:10.491348 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:12.145182 (+1653834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.367517 (+222335us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400296154. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:12.367714 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3797654400}
I20260709 12:16:12.335911 30623 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:16:12.394277 23390 rpcz_store.cc:276] 0709 12:16:11.351867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.352265 (+   398us) service_pool.cc:224] Handling call
0709 12:16:11.413562 (+ 61297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:16:11.622377 (+208815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.368139 (+745762us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54002e9354. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a9674e01 00007ff0a967383e 00007ff0ae638419 00007ff0a7bafbd4 00007ff0a7bafb62 00007ff0a7bafa91 00007ff0a7bafa0c 00007ff0a7baecc2 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:12.368292 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478443392}
I20260709 12:16:12.403208 30586 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.424211 23389 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:12.403249 30341 raft_consensus.cc:1081] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.418562 30620 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:12.438225 30625 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.404407 30565 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.455055 23315 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:12.455902 30565 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:12.440901 30651 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 12:16:12.460848 23359 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:12.468231 30622 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.469816 23314 raft_consensus.cc:1240] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:12.471413 30645 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.407434 30609 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:16:12.476336 30571 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.480135 30452 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.481372 23469 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 13: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:12.483218 30646 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.095s	user 0.002s	sys 0.000s
I20260709 12:16:12.515600 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.506098 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:12.528858 23316 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:12.513374 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.529466 23316 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:12.513803 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.506181 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.513803 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.513011 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.533867 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.537966 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.539233 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.540264 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.540952 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.530994 23390 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:12.542130 24010 raft_consensus.cc:1240] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:12.526346 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:12.528584 23464 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:16:12.546392 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.546785 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.547400 23467 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:12.510771 24007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:12.550096 24007 raft_consensus.cc:606] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:12.520649 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.552394 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.504737 30299 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:12.543419 30624 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.554407 30624 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.555682 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:12.556594 23470 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:12.558594 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.559059 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.560140 30571 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.560760 30629 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:12.560642 30571 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.561329 30629 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.562934 30571 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.564004 30629 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.565265 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:12.565457 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:12.566526 23468 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:16:12.510425 30609 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.567124 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:12.566536 30565 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:12.568287 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.568877 30565 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.569301 30565 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:12.569742 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.495841 30168 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.571095 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.571434 30168 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.494495 30503 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:12.573860 30503 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.574633 30168 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.491882 30456 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.575896 30503 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.502787 30532 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.576275 30456 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.576930 30532 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:12.577862 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 471517us
I20260709 12:16:12.578966 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.579154 30532 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.487893 30276 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.581280 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.581905 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.583447 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.583683 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:12.583676 23469 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:12.584858 30629 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:12.585490 30629 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.585902 30629 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:16:12.586181 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 73426.8us
I20260709 12:16:12.567163 30609 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.587105 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:12.587733 23463 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
W20260709 12:16:12.588873 30649 log.cc:927] Time spent T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.370s	user 0.003s	sys 0.004s
I20260709 12:16:12.589913 30462 raft_consensus.cc:993] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.590348 30462 raft_consensus.cc:1081] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.575031 30624 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.594303 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:12.595176 23387 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:12.493371 30451 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.494238 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:12.596836 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 102739us
I20260709 12:16:12.599174 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.599895 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.600970 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.601326 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.587656 23980 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.604130 30506 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:12.604545 30506 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.604890 30506 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:12.606076 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:12.606637 30452 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.605176 30609 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.611955 23469 raft_consensus.cc:1240] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:16:12.614895 23469 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:16:12.617571 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.618467 23470 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:12.619899 23391 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:12.621546 23316 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.628933 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.595824 30451 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.678339 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.678346 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.650466 24007 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:12.684167 23312 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:12.626754 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.689139 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:12.710400 23316 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.713017 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:12.713572 23390 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:12.713994 23390 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:12.714565 23390 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.715248 23390 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:12.661211 23387 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:12.623726 30330 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:12.625339 30276 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:12.724685 23387 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52266 (request call id 3128) took 1318 ms. Trace:
I20260709 12:16:12.742774 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.623553 30620 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.741984 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.745285 23387 rpcz_store.cc:276] 0709 12:16:11.406618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.445779 (+ 39161us) service_pool.cc:224] Handling call
0709 12:16:12.325680 (+879901us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b18001b2ac0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a96af646 00007ff0a95e958e 00007ff0a966af9e 00007ff0a966c7de 00007ff0a967827c 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c
0709 12:16:12.594740 (+269060us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5800852ed0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95dd571 00007ff0a95d00aa 00007ff0a9652447 00007ff0a966afe7 00007ff0a966c7de 00007ff0a967827c 00007ff0ae639b17 00007ff0a7bb46d4 00007ff0a7bb4662 00007ff0a7bb4591 00007ff0a7bb450c
0709 12:16:12.724596 (+129856us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1532997248}
I20260709 12:16:12.744995 23469 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
I20260709 12:16:12.747213 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.747846 23470 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
W20260709 12:16:12.750934 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:12.756138 23362 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:12.742524 30631 log.cc:927] Time spent T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:16:12.637544 30623 consensus_queue.cc:1048] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 12:16:12.663836 30179 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.099s
W20260709 12:16:12.750731 30657 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:16:12.767206 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:12.770524 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:12.652465 30403 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.102s
I20260709 12:16:12.629402 30532 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:12.711457 30503 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:12.731009 30035 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:16:12.780365 23434 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.763244 30341 raft_consensus.cc:493] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.782336 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:12.794191 23466 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:12.776075 23390 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:12.791500 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.688437 30533 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.772994 30456 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.799700 23316 raft_consensus.cc:3060] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:12.755333 30620 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.764663 30552 consensus_queue.cc:1059] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:16:12.798014 30452 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:12.800518 30615 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:12.855988 23319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43746 (request call id 2044) took 2060 ms. Trace:
I20260709 12:16:12.854542 23466 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:12.856549 23319 rpcz_store.cc:276] 0709 12:16:10.795747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:10.795952 (+   205us) service_pool.cc:224] Handling call
0709 12:16:10.799866 (+  3914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:16:10.800229 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:16:10.800249 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:16:10.808223 (+  7974us) log.cc:844] Serialized 26 byte log entry
0709 12:16:11.932264 (+1124041us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b18004a83c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a6569e07 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1 00007ff0a962fa0f
0709 12:16:12.455352 (+523088us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b18004a83c0. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a921c4bc 00007ff0a656a48b 00007ff0af3c49a9 00007ff0a658669b 00007ff0a658493f 00007ff0a6586c02 00007ff0a920e20b 00007ff0a9213d23 00007ff0a92140d1
0709 12:16:12.455498 (+   146us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:16:12.855082 (+399584us) log.cc:844] Serialized 14 byte log entry
0709 12:16:12.855555 (+   473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:16:12.855665 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:16:12.855721 (+    56us) raft_consensus.cc:1685] finished
0709 12:16:12.855740 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:16:12.855908 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5601,"prepare.queue_time_us":25809,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":6058,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":2575,"spinlock_wait_cycles":57554560,"thread_start_us":574829,"threads_started":2,"wal-append.queue_time_us":1266653}
I20260709 12:16:12.859282 23387 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:12.866920 23316 raft_consensus.cc:1275] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:16:12.839633 30624 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.844971 30550 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:12.801332 30503 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:12.845070 30609 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.881394 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:12.802028 30341 raft_consensus.cc:515] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:12.892891 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 204960us
I20260709 12:16:12.859740 30521 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.835853 30533 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.901324 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:12.909502 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 179053us
I20260709 12:16:12.883090 30550 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:12.860802 30629 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.891842 30539 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:12.894704 30418 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:12.921280 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:12.920418 30418 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.930174 23982 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:12.929847 30451 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.892174 30624 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.934369 30597 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:12.930220 30260 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:12.911077 30455 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:16:12.943353 30533 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.944348 30503 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:12.905222 30521 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:12.945681 30661 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:16:12.951030 30463 raft_consensus.cc:993] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.896517 30609 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.938608 30341 leader_election.cc:290] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:12.919216 30462 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:16:12.928205 30539 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:12.959795 30418 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:12.990257 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:13.053176 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 63014.4us
I20260709 12:16:12.971748 30330 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:13.007385 30609 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.010231 23313 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:16:13.011087 23318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:13.020896 30463 raft_consensus.cc:1081] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:12.979897 30569 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.048s
W20260709 12:16:12.985105 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 188829us
I20260709 12:16:13.059021 23318 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:13.024019 30624 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.060714 23318 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:13.062079 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:12.983676 30452 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.074s
I20260709 12:16:13.059607 30503 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:13.062001 30533 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:16:13.058468 30620 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.094203 23313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:13.095151 23313 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:13.095635 23313 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:13.075026 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.101382 30506 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:13.103964 30565 consensus_meta.cc:287] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.001s
I20260709 12:16:13.080523 30624 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:13.099095 30503 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:13.081550 30532 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.088384 30583 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:16:13.131991 30455 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.084492 30317 consensus_queue.cc:1059] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
W20260709 12:16:13.158564 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 248103us
I20260709 12:16:13.223902 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.159322 30629 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.221331 23313 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:13.220366 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 145470us
I20260709 12:16:13.240224 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.219661 23318 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.296078 23391 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:13.264773 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.296296 23319 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:16:13.239699 30504 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:13.231205 30670 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.232327 23394 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
W20260709 12:16:13.240398 30669 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:16:13.306018 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:13.307353 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:13.311107 30670 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:13.312580 30658 log.cc:927] Time spent T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.498s	user 0.014s	sys 0.000s
I20260709 12:16:13.318477 23318 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.313958 30670 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.294646 23362 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:13.328317 23389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.015s	sys 0.033s
I20260709 12:16:13.337508 30637 consensus_queue.cc:1048] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:13.337567 30403 raft_consensus.cc:3055] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:13.339418 30403 raft_consensus.cc:740] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:13.340320 30403 consensus_queue.cc:260] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:13.342080 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47524.5us
I20260709 12:16:13.343369 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.345623 30504 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:13.346015 30521 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.346144 30504 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:13.346628 30622 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.347599 30671 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:13.347208 30504 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.348014 30671 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:13.348520 30671 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:13.350386 30179 raft_consensus.cc:993] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:13.350883 30179 raft_consensus.cc:1081] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:13.352610 23321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:13.353165 23321 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:13.353554 23321 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:13.356068 30403 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:13.359768 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.239166 30317 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:13.231211 30455 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.285457 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.291630 30452 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.199991 30565 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.297937 30652 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:13.223661 30532 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.364578 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.365872 23312 raft_consensus.cc:1240] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:16:13.367534 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 82251.5us
W20260709 12:16:13.384132 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 378413us
I20260709 12:16:13.386124 23469 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:16:13.389245 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.389467 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.391330 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.385392 23321 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.392846 23391 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:16:13.393280 23316 raft_consensus.cc:1240] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:13.231277 30609 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.401063 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7238eb389343cd954cb756bed5925f" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.223232 30628 raft_consensus.cc:493] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.372864 30622 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:13.329962 30539 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.367128 30317 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:13.447949 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.448343 30317 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.409268 23313 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.450230 30317 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:13.439921 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:13.328734 23389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.015s	sys 0.033s
W20260709 12:16:13.440845 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56030.8us
I20260709 12:16:13.443672 23463 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:13.452945 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.444375 24009 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:16:13.444325 23321 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.453351 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.445363 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
W20260709 12:16:13.461993 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 52332.1us
I20260709 12:16:13.467404 30532 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:13.469874 30565 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.471151 30645 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.471750 30645 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.472436 30455 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.474344 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.475821 30557 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:13.377265 30035 consensus_queue.cc:1059] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:16:13.476060 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.476392 30557 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.477744 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.478222 30645 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.479265 30569 raft_consensus.cc:3055] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:16:13.479660 30569 raft_consensus.cc:740] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
W20260709 12:16:13.479805 30597 consensus_queue.cc:1175] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2778b449761a46708f2f643b1b5b5854" responder_term: 16 status { last_received { term: 12 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 15. Current term is 16. Ops: [15.12-15.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:16:13.480473 30569 consensus_queue.cc:260] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.481153 23321 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:13.382572 30652 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.375842 30452 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.482546 23470 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:13.483286 23464 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:16:13.483546 30652 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.484380 30452 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.441018 30624 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.357820 30503 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.485725 30624 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.487838 30624 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.447949 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.489153 23387 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:13.489625 30506 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.442416 30628 raft_consensus.cc:515] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.449302 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:13.490540 23462 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:13.446357 23395 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:13.492239 23362 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.483165 30462 consensus_queue.cc:1059] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:13.491714 30628 leader_election.cc:290] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:13.447127 30668 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:16:13.506421 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.494634 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.494805 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.495926 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:13.497237 23287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.150.195:33909 timed out after 1.609s (SENT) [suppressed 5 similar messages]
W20260709 12:16:13.445592 30676 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:16:13.513010 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.513849 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.511096 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.516819 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.517686 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a658e02588314c9ca8a73f5eb423fb27" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.522181 23465 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:16:13.525784 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.528779 23462 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
W20260709 12:16:13.512164 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 216303us
I20260709 12:16:13.537190 23362 leader_election.cc:304] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.539853 23462 raft_consensus.cc:1240] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 10. Current term is 11. Ops: []
I20260709 12:16:13.541112 23362 leader_election.cc:304] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:13.535457 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 64689.2us
I20260709 12:16:13.505087 30506 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.547844 23313 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:16:13.548250 23313 pending_rounds.cc:85] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 11
I20260709 12:16:13.548465 23313 pending_rounds.cc:107] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:16:13.548705 23313 raft_consensus.cc:2889] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:13.550366 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.548723 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.549727 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:13.520035 30569 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:13.567148 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:13.566541 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69289us
I20260709 12:16:13.569976 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.570245 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.571817 23315 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:13.574108 23318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.574512 23317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.564159 23436 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:13.574977 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.578598 23395 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
W20260709 12:16:13.581876 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:13.583312 23387 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.587177 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.580849 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.542810 30226 raft_consensus.cc:2749] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:13.589161 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.592312 23395 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:13.584673 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.594624 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.565673 30341 raft_consensus.cc:2749] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.596004 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:13.576655 30615 consensus_meta.cc:287] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.715s	user 0.004s	sys 0.008s
I20260709 12:16:13.596014 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.600610 23360 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.601342 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.550262 30575 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:16:13.586328 30456 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:13.606314 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.612737 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.622766 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.623842 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.625232 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.598856 30609 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:13.629753 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.631261 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.600356 30565 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.601668 30615 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:13.607656 30622 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.626005 30532 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:13.659269 30557 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.625602 30226 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:13.629462 30521 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:13.638938 30609 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:13.629765 30539 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:13.655233 30503 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:13.638058 30533 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.663054 23467 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:16:13.665836 30521 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.665977 30609 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:16:13.666267 30330 consensus_meta.cc:287] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Time spent flushing consensus metadata: real 0.605s	user 0.009s	sys 0.004s
I20260709 12:16:13.643297 30276 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:13.646806 30629 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:16:13.668985 30418 consensus_meta.cc:287] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: Time spent flushing consensus metadata: real 0.610s	user 0.005s	sys 0.005s
I20260709 12:16:13.653861 30685 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:13.648799 30438 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:16:13.688043 30671 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.692301 23389 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:16:13.693095 30629 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.693769 30629 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:13.696816 30671 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.701772 30330 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:13.708204 30418 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.711228 24010 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 12. Current term is 13. Ops: []
I20260709 12:16:13.703006 30533 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.672868 30521 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:16:13.716902 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69767us
I20260709 12:16:13.721788 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:13.725657 30533 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:13.726162 30615 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.726684 30615 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.728808 30615 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.730572 30645 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.731083 30645 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.732999 30645 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:13.746119 30418 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.741920 30557 consensus_queue.cc:1059] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:16:13.735914 30533 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:13.758579 30677 consensus_queue.cc:1059] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:13.759683 30677 raft_consensus.cc:3055] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:13.760046 30677 raft_consensus.cc:740] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:13.760780 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.761322 30563 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:13.761711 30563 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:13.762452 30563 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.763685 30563 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:13.757305 30533 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.760932 30677 consensus_queue.cc:260] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.765610 24008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.767002 30677 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:13.767515 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:13.767897 30686 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
I20260709 12:16:13.768760 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.769351 23465 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:13.770562 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:13.771019 30681 log.cc:927] Time spent T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:16:13.772955 23465 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:13.774547 24009 raft_consensus.cc:2410] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 12:16:13.778393 23284 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:13.779713 30609 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.782208 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.782876 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.783368 30517 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:13.783766 30517 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:13.784089 30517 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:13.788327 30629 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.790558 30629 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.793964 30533 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:13.798403 30670 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:13.798852 30670 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:13.799360 30670 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:16:13.802039 23981 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:13.805029 23314 raft_consensus.cc:1240] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:13.813658 24007 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 11.
I20260709 12:16:13.815886 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.825109 30674 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:16:13.844373 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.859087 24013 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:13.817575 30623 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:16:13.866933 30645 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:13.813169 30521 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:13.870810 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 239613us
I20260709 12:16:13.827842 30403 consensus_queue.cc:1059] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:16:13.863127 30615 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:13.856935 30609 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.878203 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.881613 23312 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:16:13.871622 30456 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.887225 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.893592 30539 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:13.880015 30615 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.899143 30456 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:13.900233 30615 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:13.884368 30533 consensus_queue.cc:1059] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:16:13.902468 30533 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:16:13.902869 30533 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:13.903697 30533 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.904984 30533 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:13.907317 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.912089 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:13.914711 23287 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.910356 30550 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:13.917565 30550 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:13.917724 30276 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:16:13.918977 30503 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:13.924804 30448 consensus_queue.cc:1059] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:13.928297 30539 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:13.891830 30629 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:13.944314 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.944640 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:13.981173 23360 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:13.983578 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.983335 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:13.995007 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:13.995664 23388 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:13.995007 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:13.997421 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:13.999122 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:13.999788 23391 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:13.911303 30456 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:13.998359 23468 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:14.006211 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.006441 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:14.007663 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:13.966917 30276 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:16:14.028458 23286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.029713 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.030202 23464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:14.035771 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:14.036319 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:14.031872 23434 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.039938 23395 raft_consensus.cc:1240] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 21. Current term is 22. Ops: [19.15-21.16]
I20260709 12:16:14.040854 23394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:14.041376 23390 raft_consensus.cc:1240] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:16:14.041733 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:14.042520 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:14.042752 23467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:13.995702 30539 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:14.048240 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.050534 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:14.041407 23434 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.044803 23470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:14.054592 23389 raft_consensus.cc:2393] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 20: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:13.961880 30517 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.045885 23466 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:16:13.999235 30504 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:14.046522 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:14.095460 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 54130us
I20260709 12:16:14.047937 30539 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:13.995738 30448 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:16:13.996855 30695 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:16:14.045050 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.000275 30629 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 12:16:14.096309 23434 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:14.103919 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54865.8us
I20260709 12:16:14.104130 23388 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 23.
I20260709 12:16:14.099525 30539 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:14.114605 24013 raft_consensus.cc:2410] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
W20260709 12:16:14.099053 30692 log.cc:927] Time spent T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:16:14.060904 30276 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
W20260709 12:16:14.122782 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 77806.8us
I20260709 12:16:14.123947 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.096719 30679 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:16:14.048331 30622 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:16:14.138170 24007 raft_consensus.cc:3055] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:14.138550 24007 raft_consensus.cc:740] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.128825 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.139771 23434 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.140849 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.134370 30517 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:14.115698 30675 log.cc:927] Time spent T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:16:14.108942 30615 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:14.139434 24007 consensus_queue.cc:260] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.155186 24007 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:14.135752 30503 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:14.149962 30456 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:14.165802 23468 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 23.
I20260709 12:16:14.158993 30521 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:14.159541 30679 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:14.164372 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:14.186160 23465 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:14.163448 30666 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.192559 24007 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 23.
W20260709 12:16:14.185359 30704 log.cc:927] Time spent T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:16:14.183867 30695 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:16:14.185532 30652 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:14.207810 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 84007.3us
I20260709 12:16:14.183323 30678 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:14.208631 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.215462 23287 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.216603 23465 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
W20260709 12:16:14.202363 30557 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 12:16:14.212296 30622 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:14.196607 30666 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.222298 30695 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:14.222847 30695 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.223825 30666 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:14.223666 30695 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.232232 24012 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:14.243243 30490 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:14.227679 30677 raft_consensus.cc:3055] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:14.184413 30276 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:14.246547 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 105773us
I20260709 12:16:14.243844 30490 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:14.253917 30708 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:16:14.257817 23393 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:14.268764 24013 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:16:14.269310 24013 pending_rounds.cc:85] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting all ops after (but not including) 15
I20260709 12:16:14.269564 24013 pending_rounds.cc:107] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:16:14.269785 24013 raft_consensus.cc:2889] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:14.268810 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:14.281347 23362 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:14.285004 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:14.286085 24007 raft_consensus.cc:2393] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 21: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:14.250567 23434 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.292047 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.292407 23393 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:14.295168 23320 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:16:14.296243 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.296939 23391 raft_consensus.cc:2468] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:16:14.298729 30539 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.249366 30677 raft_consensus.cc:740] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.300834 30539 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:14.301301 30677 consensus_queue.cc:260] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.302357 30677 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:14.303241 23314 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:14.305576 30674 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:14.306195 30674 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:14.307134 30674 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.309520 30711 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:16:14.309903 23436 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:14.310876 30680 raft_consensus.cc:2804] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:14.311380 30680 raft_consensus.cc:697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:14.312145 30680 consensus_queue.cc:237] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.283288 30490 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:14.324075 30698 log.cc:927] Time spent T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:16:14.328243 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:14.330454 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.331293 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:14.331789 23388 raft_consensus.cc:1240] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:16:14.337582 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.339816 30341 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:14.346307 30503 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:16:14.349153 23982 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:14.352654 30035 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:14.363581 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 73638.2us
I20260709 12:16:14.364620 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87
W20260709 12:16:14.376175 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 125691us
I20260709 12:16:14.384546 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:14.363289 30670 consensus_meta.cc:287] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.002s
I20260709 12:16:14.387382 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.385393 30652 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:14.383329 30588 raft_consensus.cc:3055] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:14.401808 30664 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:14.403959 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.403250 30711 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:16:14.389061 30276 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:14.421265 23287 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:14.384159 30403 consensus_queue.cc:1059] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:16:14.388024 30623 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:16:14.433194 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:14.431473 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.406104 30588 raft_consensus.cc:740] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.396584 30503 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:16:14.438472 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:14.422963 30664 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:14.420706 30276 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:14.440076 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.440229 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.423707 30685 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:16:14.443254 30711 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:14.440734 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.397131 30670 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.463438 30670 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:14.464160 23461 raft_consensus.cc:1240] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:16:14.452117 30276 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:14.465549 23313 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 14. Current term is 15. Ops: []
I20260709 12:16:14.437191 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.443020 23395 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:14.441408 23434 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.439607 23391 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:16:14.469094 30550 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:14.482357 23316 raft_consensus.cc:1240] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 16. Current term is 17. Ops: []
I20260709 12:16:14.483104 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:14.485570 23462 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:16:14.485761 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:14.488405 23466 raft_consensus.cc:1240] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 12:16:14.494637 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54652.2us
I20260709 12:16:14.495594 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.497455 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.499984 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.476466 30341 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:14.494105 30695 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:16:14.503130 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.506846 30677 raft_consensus.cc:3055] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:16:14.509752 30539 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:14.502254 30565 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:14.519930 30179 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:16:14.520007 30637 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.475818 30711 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.501206 30503 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:14.494240 30597 consensus_queue.cc:1059] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:16:14.561230 30179 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.494462 30715 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:14.494830 30670 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:16:14.561555 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 58511us
W20260709 12:16:14.567911 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 149893us
I20260709 12:16:14.558773 30677 raft_consensus.cc:740] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.567804 30695 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:14.592581 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.567989 30715 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:14.593844 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:14.597744 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.599887 30532 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:14.600623 30532 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.601642 30532 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.608250 30276 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.611799 30677 consensus_queue.cc:260] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.614293 23316 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 21. Current term is 22. Ops: []
I20260709 12:16:14.615056 23434 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.595338 30539 consensus_queue.cc:1059] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:16:14.618407 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:14.618297 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.620236 23395 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:16:14.609980 30711 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.625029 30670 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:14.622926 23434 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:14.628563 30670 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:16:14.593022 30565 consensus_queue.cc:1059] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:16:14.574188 30533 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:16:14.567356 30637 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.572907 30179 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.635814 30179 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:14.641801 30539 consensus_queue.cc:1243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer da1e2fd157894634bf92d6c82c06ec64 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:16:14.642481 30539 raft_consensus.cc:3055] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:14.642824 30539 raft_consensus.cc:740] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.643703 30539 consensus_queue.cc:260] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.645016 30539 raft_consensus.cc:3060] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:14.622649 30695 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:16:14.651331 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 54660.1us
I20260709 12:16:14.654481 30715 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.655988 30715 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:14.658239 30565 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:14.658828 30565 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.659641 30565 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.660833 30565 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:14.613164 30677 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:14.668200 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 49968.1us
I20260709 12:16:14.680334 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.685750 24010 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:14.674243 30622 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:16:14.685133 30678 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.709836 30678 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.712136 30678 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:14.687633 30452 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:16:14.678369 30503 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:16:14.703531 30276 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:14.724108 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
W20260709 12:16:14.741809 30728 log.cc:927] Time spent T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.142s	user 0.004s	sys 0.000s
I20260709 12:16:14.741940 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:14.687227 30533 consensus_queue.cc:1059] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:16:14.726085 30677 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:14.743666 30276 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:16:14.759104 30448 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:14.759801 30677 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.759157 30533 raft_consensus.cc:3055] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:14.760983 30533 raft_consensus.cc:740] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:14.760221 30448 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:16:14.761592 30448 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:14.762187 30677 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:14.762992 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.763621 23316 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:14.765829 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:14.769778 30691 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:14.771889 30691 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:14.773577 30691 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:14.778187 30533 consensus_queue.cc:260] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.779548 30533 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:14.762655 30448 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.783372 30448 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:14.805701 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 184020us
W20260709 12:16:14.807941 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 188675us
I20260709 12:16:14.814631 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:14.817499 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:14.814631 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:14.818971 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:14.818806 23388 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:14.827598 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.790233 30691 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.794348 30637 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:14.829613 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:14.801685 30588 consensus_queue.cc:260] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:14.834443 23391 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 12:16:14.831565 30730 log.cc:927] Time spent T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.050s	user 0.000s	sys 0.004s
I20260709 12:16:14.836017 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:14.836220 30539 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:14.847455 23466 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:16:14.841230 30624 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:14.856760 30624 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:14.840404 30695 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:16:14.853143 30711 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:16:14.857318 30624 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:14.852365 30588 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:14.866737 30624 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:14.877233 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:14.867475 23362 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:14.878917 23319 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:16:14.899986 24012 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:16:14.911178 23319 pending_rounds.cc:85] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 13
I20260709 12:16:14.911513 23319 pending_rounds.cc:107] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:16:14.911743 23319 raft_consensus.cc:2889] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:14.893190 23314 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:16:14.912268 23321 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:14.913586 23314 pending_rounds.cc:85] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting all ops after (but not including) 10
I20260709 12:16:14.913902 23314 pending_rounds.cc:107] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:16:14.914183 23314 raft_consensus.cc:2889] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:16:14.926085 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 58693.2us
I20260709 12:16:14.919308 30711 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:16:14.933794 23461 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 21. Current term is 22. Ops: []
I20260709 12:16:14.935400 30711 consensus_queue.cc:1059] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:14.920796 30678 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:16:14.944823 23321 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:14.946972 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:14.943166 30504 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:14.954919 30677 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:14.967773 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:14.953862 30691 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:14.976066 23217 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 12 to 15, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 15 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:14.978485 23217 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:14.959280 30341 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:16:14.977355 30677 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:15.020553 23217 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 17 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 19 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.028091 30624 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:15.045769 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:15.054499 23395 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:15.061870 30695 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:16:15.062395 30695 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:15.070371 30695 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:15.071798 30695 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:15.074172 23465 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:15.075309 30677 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.083425 30673 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.083997 30673 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.086053 30673 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.091634 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.093813 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:15.100548 23388 raft_consensus.cc:1275] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:16:15.107036 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.107774 23313 raft_consensus.cc:2368] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:16:15.116447 23217 catalog_manager.cc:5697] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 14 to 17, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.118733 23217 catalog_manager.cc:5697] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.114439 23436 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:15.120740 23217 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.130028 23360 leader_election.cc:400] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:16:15.130746 23360 leader_election.cc:403] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:15.124269 30624 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:15.127873 30452 consensus_queue.cc:1048] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:16:15.157115 23461 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:16:15.157119 30504 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:16:15.158659 30723 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:15.164829 23217 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 24, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 24 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:15.132182 30533 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:16:15.173111 23217 catalog_manager.cc:5697] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:15.178349 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45681.5us
I20260709 12:16:15.171651 30711 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:16:15.184527 23392 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:15.187315 24010 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:16:15.205544 30565 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:15.211169 23215 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 347) took 10601 ms. Trace:
I20260709 12:16:15.218070 24013 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:16:15.211690 23215 rpcz_store.cc:276] 0709 12:16:04.627351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:04.645128 (+ 17777us) service_pool.cc:224] Handling call
0709 12:16:15.211066 (+10565938us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:12.437496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:12.445395 (+  7899us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:12.445446 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:16:12.445466 (+    20us) tablet.cc:662] Decoding operations
0709 12:16:12.447405 (+  1939us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:12.447467 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:12.447495 (+    28us) tablet.cc:685] Acquiring locks for 29 operations
0709 12:16:12.609927 (+162432us) tablet.cc:701] Row locks acquired
0709 12:16:12.609944 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:16:12.610091 (+   147us) write_op.cc:270] Start()
0709 12:16:12.610203 (+   112us) write_op.cc:276] Timestamp: P: 1783599372610038 usec, L: 0
0709 12:16:12.610226 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:16:12.741220 (+130994us) log.cc:844] Serialized 11444 byte log entry
0709 12:16:13.609514 (+868294us) op_driver.cc:464] REPLICATION: finished
0709 12:16:13.926155 (+316641us) write_op.cc:301] APPLY: starting
0709 12:16:13.926257 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:14.316992 (+390735us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:14.317136 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:14.551183 (+234047us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:14.554556 (+  3373us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=58,mrs_lookups=0
0709 12:16:14.560205 (+  5649us) write_op.cc:312] APPLY: finished
0709 12:16:14.688836 (+128631us) log.cc:844] Serialized 252 byte log entry
0709 12:16:14.750049 (+ 61213us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:14.750281 (+   232us) write_op.cc:454] Released schema lock
0709 12:16:14.750346 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:14.750998 (+   652us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:14.751028 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2015795,"reported_tablets":48,"spinlock_wait_cycles":1127808,"child_traces":[["op",{"apply.queue_time_us":299587,"apply.run_cpu_time_us":51733,"apply.run_wall_time_us":824828,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125463,"num_ops":29,"prepare.queue_time_us":145217,"prepare.run_cpu_time_us":24914,"prepare.run_wall_time_us":530645,"raft.queue_time_us":212783,"raft.run_cpu_time_us":22350,"raft.run_wall_time_us":275622,"replication_time_us":988129,"thread_start_us":691757,"threads_started":5,"wal-append.queue_time_us":462348,"wal-append.run_cpu_time_us":38477,"wal-append.run_wall_time_us":1445683}]]}
I20260709 12:16:15.219684 30532 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:15.193794 23217 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:15.226369 23217 catalog_manager.cc:5697] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from <none> to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.230055 23217 catalog_manager.cc:5697] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:15.246114 30738 log.cc:927] Time spent T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:16:15.227936 30711 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
W20260709 12:16:15.246253 30737 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:16:15.269820 30583 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.230654 30532 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.270388 30583 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.270560 30532 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.272536 30532 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.267928 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 12 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 20 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.278801 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:15.279429 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:15.279834 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.280671 23314 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:16:15.279929 23217 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.284394 23217 catalog_manager.cc:5697] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.279448 23465 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 13.
I20260709 12:16:15.286856 23217 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.283988 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:15.292481 23217 catalog_manager.cc:5697] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 16 to 17, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.299014 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.297245 23217 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 17, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.304737 23217 catalog_manager.cc:5697] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 18 to 22, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 22 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.308127 23217 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 19, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:15.272538 30583 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:15.296489 30693 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:15.318807 24007 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:15.321049 23217 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 18 to 23, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 23 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:15.325002 23217 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:15.330133 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:15.331125 30532 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:16:15.331596 30532 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.329604 23217 catalog_manager.cc:5697] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 18 to 23. New cstate: current_term: 23 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.319912 30693 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.334393 30504 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 16. Current term is 17.
I20260709 12:16:15.345757 30693 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:15.334823 23217 catalog_manager.cc:5697] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 17, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.354323 23217 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:15.333922 30532 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:15.393594 30588 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:15.406703 30588 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.414829 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.415388 23312 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:15.417732 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:15.421842 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:15.422765 23388 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:15.413167 30588 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:15.420585 30179 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:15.445780 30179 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:15.447255 30179 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:15.454200 30712 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.456982 30712 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.459024 30712 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.478120 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:15.478914 23395 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 21: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:15.486688 30532 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.513885 23312 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:15.520295 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.520871 24010 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:15.525967 30179 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.527876 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:15.528676 23391 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:15.529811 30452 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:15.530191 30452 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.530562 30452 raft_consensus.cc:3060] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:15.545217 30179 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:15.545518 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:15.546355 23312 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:15.555152 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:15.555635 30671 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.555797 23391 raft_consensus.cc:3060] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:15.556185 30671 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.558252 30671 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.522286 30677 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:16:15.560936 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.561602 24010 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:15.565909 23391 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:15.566849 23312 raft_consensus.cc:2468] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:15.568096 23980 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:15.570513 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:15.569419 30035 raft_consensus.cc:2804] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:15.572804 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.573067 30035 raft_consensus.cc:697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:15.573967 30035 consensus_queue.cc:237] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.581380 30671 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:15.611255 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:15.627477 23461 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:15.567621 30693 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.571187 30179 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:16:15.626667 30671 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.653891 30671 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:15.655233 30693 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:15.657215 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:15.661589 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:15.665935 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:15.667027 23320 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 14: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:15.700718 30452 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.703246 30452 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.704946 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:15.718000 23461 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 14.
I20260709 12:16:15.719286 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:15.730816 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:15.738670 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:15.745055 23388 raft_consensus.cc:2393] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 21: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:15.734979 24010 raft_consensus.cc:2468] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:15.746412 23436 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:15.735378 30711 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:15.755664 30711 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:15.748389 30697 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:16:15.756583 30711 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.749711 30712 raft_consensus.cc:2804] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:15.759284 30712 raft_consensus.cc:697] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 21 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:15.761541 30712 consensus_queue.cc:237] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.770713 30759 tablet_replica.cc:333] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:15.773487 30671 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.802475 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:15.808215 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 70636.3us
W20260709 12:16:15.815661 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70823.4us
I20260709 12:16:15.843694 30452 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.845423 30452 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.850471 30532 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 14 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.851811 30452 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.854903 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:15.856567 24013 raft_consensus.cc:2393] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 2778b449761a46708f2f643b1b5b5854 in current term 14: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:15.861932 30671 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.888453 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:15.899884 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.900731 30624 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.901520 23314 raft_consensus.cc:2393] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 17: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:15.901779 30624 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:15.922142 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:15.923442 23320 raft_consensus.cc:2393] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:15.939682 30624 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.946103 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:15.947371 23313 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:15.950261 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.951156 24007 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:15.953644 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:15.954583 24012 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:15.918648 30452 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.963661 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:15.966722 30712 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:15.967293 30712 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:15.967790 30712 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:15.984927 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:15.979530 30452 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.005738 30438 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.006671 30438 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.010144 30623 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.011152 30623 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.015018 30673 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:16.015889 30673 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.020838 30623 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.022094 30673 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:16.024055 30438 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:15.999161 30624 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:16.040263 30452 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.041726 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.048834 24012 raft_consensus.cc:2410] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:16:16.043717 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.051699 24007 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:16:16.033617 30624 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:16.053259 30624 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:16.062482 23313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:16.064468 23313 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:16:16.109874 30764 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:16.111188 30764 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.114363 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:16.115427 23314 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:16.121656 24013 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:16.125602 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.120427 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:16.126385 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:16.127148 30673 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:16.127609 30673 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:16.128000 30673 raft_consensus.cc:3060] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:16.131435 23391 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:16.128440 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.135085 23392 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 22: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:16.135614 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:16.146548 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.144114 30764 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:16.155438 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:16.162563 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.165386 23465 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:16.165916 23320 raft_consensus.cc:1275] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:16.165796 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.161775 30764 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:16.169847 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.180207 30693 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:16:16.180908 30539 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.181958 30539 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.170449 30764 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:16.174191 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.188740 30712 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.200977 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.213231 30766 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:16.213973 30766 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:16.214519 30766 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:16.218645 30539 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.219364 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.220148 24013 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:16.221390 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:16.222299 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.223162 23465 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:16.187490 30764 raft_consensus.cc:3060] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:16.236289 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 78555.1us
W20260709 12:16:16.237007 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46606.8us
I20260709 12:16:16.251122 30693 consensus_queue.cc:1048] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:16.251637 30759 raft_consensus.cc:2243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:16.253351 30532 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.254135 30532 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.256436 30532 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:16.256968 30532 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:16.257897 30532 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.258383 30532 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:16.259761 24013 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:16.264231 30711 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:16:16.256845 30624 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.297159 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.304873 30712 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.306725 23462 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:16.307857 30759 raft_consensus.cc:2272] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:16.309720 30671 raft_consensus.cc:670] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 12:16:16.316370 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:16.313087 30532 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.321542 30532 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.323102 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:16.302114 30438 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:16.325896 30438 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:16.327092 30438 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.372611 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.373312 23469 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 22.
I20260709 12:16:16.375589 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:16.375710 23469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:16.376310 23469 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:16.377091 30742 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:16.376896 30711 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:16.377506 30742 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.377892 30742 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:16.387130 24012 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:16.388871 30767 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:16.389310 23392 raft_consensus.cc:1275] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:16.392352 30742 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.396632 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:16.400825 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:16.400851 30742 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:16.401648 23462 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:16.410799 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:16.411929 23320 raft_consensus.cc:2368] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18.
I20260709 12:16:16.413180 30759 ts_tablet_manager.cc:1916] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:16.432186 24007 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:16.434844 30766 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:16.436628 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 48134.6us
W20260709 12:16:16.440431 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 136000us
I20260709 12:16:16.459787 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:16.442868 30673 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.451056 23434 leader_election.cc:400] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18.
I20260709 12:16:16.463305 23434 leader_election.cc:403] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:16:16.456281 23462 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 23.
I20260709 12:16:16.474867 23469 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:16.472105 30671 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:16.493243 30624 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.500844 23287 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:16.505066 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:16.490301 30452 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055) with higher term. Message: Invalid argument: T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for earlier term 17. Current term is 18.
I20260709 12:16:16.509076 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:16.510107 23320 raft_consensus.cc:2393] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 15: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:16.507601 30742 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:16.511425 30532 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:16.506349 30764 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.537669 30764 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.539422 23461 raft_consensus.cc:2410] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:16:16.541291 30673 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:16.551913 30769 log.cc:927] Time spent T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.069s	user 0.000s	sys 0.004s
W20260709 12:16:16.563288 30768 log.cc:927] Time spent T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:16:16.591540 30712 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.592489 30712 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:16.595695 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 49763.3us
I20260709 12:16:16.598119 30766 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:16.599948 30742 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:16.601384 30742 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:16.605216 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 51523.8us
I20260709 12:16:16.608596 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:16.609494 23312 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:16:16.608623 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:16.610905 23314 raft_consensus.cc:2393] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 16: Already voted for candidate 2778b449761a46708f2f643b1b5b5854 in this term.
I20260709 12:16:16.613536 30711 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.614244 30711 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.618268 30532 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:16.620071 30532 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.595708 30712 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:16.654138 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.654877 24013 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 22.
I20260709 12:16:16.659421 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:16.660167 30539 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:16.660553 30539 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.660903 30539 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:16.617050 30711 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.671765 24013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:16.672875 24013 raft_consensus.cc:606] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.673533 24013 raft_consensus.cc:614] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:16:16.709630 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 235366us
I20260709 12:16:16.724462 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:16.728924 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.737586 30503 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.738231 30503 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.740546 30503 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.742599 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:16.743472 24013 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:16.725194 23392 raft_consensus.cc:606] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.747884 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.750854 23461 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:16.747363 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
W20260709 12:16:16.764091 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 257890us
I20260709 12:16:16.764282 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:16.784950 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:16.784950 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:16.764597 30503 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:16.799942 23362 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:16.815385 24007 raft_consensus.cc:2468] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:16.817775 24013 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:16.799254 30503 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:16.782119 30712 consensus_queue.cc:1048] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:16:16.829990 30715 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:16.823556 30503 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:16.835908 23320 raft_consensus.cc:2468] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 16.
I20260709 12:16:16.858896 30539 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.872875 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:16.874020 24013 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:16.875463 30685 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:16.895296 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.896204 23395 raft_consensus.cc:2410] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:16:16.900381 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:16.901059 24010 raft_consensus.cc:606] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.903116 30759 ts_tablet_manager.cc:1929] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 12:16:16.903771 30759 log.cc:1199] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/ec4e3b5eed804f2a8c2a01e5d1ae5f53
I20260709 12:16:16.904497 23267 placement_policy_util.cc:248] tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 12:16:16.905535 30759 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33622 (request call id 0) took 1272 ms. Trace:
I20260709 12:16:16.906352 30759 rpcz_store.cc:276] 0709 12:16:15.659862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:15.661167 (+  1305us) service_pool.cc:224] Handling call
0709 12:16:15.770330 (+109163us) ts_tablet_manager.cc:1151] Deleting tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53
0709 12:16:15.770429 (+    99us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:16:16.681143 (+910714us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:16.902974 (+221831us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:16.905389 (+  2415us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":523422,"tablet-delete.queue_time_us":107671,"thread_start_us":164100,"threads_started":1}
I20260709 12:16:16.892701 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19b18a93b6e8478e9f44ce88b342ab54" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:16.912523 23392 raft_consensus.cc:2393] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 22: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:16.917924 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:16.918793 24010 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.861326 30539 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:16.876117 30685 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.924729 30685 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:16.925817 24010 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:16.926610 24010 raft_consensus.cc:606] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.928215 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:16.930222 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:16.930111 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:16.931095 30767 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:16.931360 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:16.931631 30767 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:16.932541 30767 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.936479 23436 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:16.937947 30765 raft_consensus.cc:2804] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:16.938643 30765 raft_consensus.cc:697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:16.939778 30765 consensus_queue.cc:237] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:16.942549 30742 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:16.943120 30742 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:16.946724 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:16.946802 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:16.947685 23468 raft_consensus.cc:2393] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 16: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:16.949147 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:16.952247 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:16.970708 23465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:16.970790 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:16.974395 23465 raft_consensus.cc:2368] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 22. Current term is 23.
I20260709 12:16:16.978529 23395 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:16.994135 24013 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 23.
I20260709 12:16:17.019433 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:17.022622 23360 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:17.030884 30697 raft_consensus.cc:993] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.030309 23214 catalog_manager.cc:5697] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 18 to 20, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 20 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.031364 30697 raft_consensus.cc:1081] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.033593 23214 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.043779 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:17.044728 23461 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:17.048173 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:17.025247 30711 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:16:17.030189 30742 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:17.004031 30503 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.057176 23362 leader_election.cc:400] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 22. Current term is 23.
I20260709 12:16:17.068363 23362 leader_election.cc:403] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:17.071872 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:17.046861 23312 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:17.087872 23312 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:17.088328 23312 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.097705 30776 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:17.100448 23214 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:17.101647 30723 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:16:17.104983 30260 raft_consensus.cc:993] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.105561 30260 raft_consensus.cc:1081] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.073009 30711 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.097559 30770 raft_consensus.cc:2804] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:16:17.117628 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69679us
I20260709 12:16:17.122912 23461 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:17.128137 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:17.130513 30784 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:17.130653 30783 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:17.131054 30784 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:17.131431 30784 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:17.133395 30711 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.134553 30503 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:17.138803 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:17.146639 30767 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:17.147087 23314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:17.147249 30767 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:17.149605 30767 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.142119 23312 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.128157 30770 raft_consensus.cc:697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:17.175194 30770 consensus_queue.cc:237] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.183955 23312 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:17.185729 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:17.186370 23461 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.193493 23465 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:17.194017 30784 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.196946 30783 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:17.198921 23214 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 13 to 17, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.200486 30784 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:17.204093 30691 raft_consensus.cc:993] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.204607 30691 raft_consensus.cc:1081] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.206910 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:17.207604 23395 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:17.209542 23312 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:17.210152 23312 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:17.210503 23312 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.214093 23461 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:17.215301 23217 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 224) took 5240 ms. Trace:
I20260709 12:16:17.201251 23214 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 17 to 18, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.222184 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:17.225847 30783 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:16:17.226601 30783 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.226732 30691 raft_consensus.cc:993] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.227721 30783 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.233682 23214 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 15 to 17, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:17.235193 30723 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909) with higher term. Message: Invalid argument: T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 for earlier term 22. Current term is 23.
I20260709 12:16:17.215672 23217 rpcz_store.cc:276] 0709 12:16:11.967610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:11.968080 (+   470us) service_pool.cc:224] Handling call
0709 12:16:12.472867 (+504787us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a65500b2 00007ff0af4c9759 00007ff0af461a76 00007ff0af432d58 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc
0709 12:16:17.208249 (+4735382us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:15.537558 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:15.543657 (+  6099us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:15.543693 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:16:15.543710 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:15.545414 (+  1704us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:15.545520 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:15.545551 (+    31us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:16:15.547388 (+  1837us) tablet.cc:701] Row locks acquired
0709 12:16:15.547405 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:16:15.547541 (+   136us) write_op.cc:270] Start()
0709 12:16:15.547657 (+   116us) write_op.cc:276] Timestamp: P: 1783599375547494 usec, L: 0
0709 12:16:15.547677 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:16:15.571918 (+ 24241us) log.cc:844] Serialized 9478 byte log entry
0709 12:16:15.751569 (+179651us) op_driver.cc:464] REPLICATION: finished
0709 12:16:15.780169 (+ 28600us) write_op.cc:301] APPLY: starting
0709 12:16:15.870176 (+ 90007us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:16.061671 (+191495us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:16.061814 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:16.492834 (+431020us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:16.493241 (+   407us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 12:16:16.493318 (+    77us) write_op.cc:312] APPLY: finished
0709 12:16:16.538107 (+ 44789us) log.cc:844] Serialized 212 byte log entry
0709 12:16:16.538501 (+   394us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:16.539185 (+   684us) write_op.cc:454] Released schema lock
0709 12:16:16.539310 (+   125us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:16.540027 (+   717us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:16.540065 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":335914,"reported_tablets":38,"spinlock_wait_cycles":87875712,"child_traces":[["op",{"apply.queue_time_us":20624,"apply.run_cpu_time_us":46052,"apply.run_wall_time_us":759762,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104543,"num_ops":24,"prepare.queue_time_us":161844,"prepare.run_cpu_time_us":5968,"prepare.run_wall_time_us":68925,"replication_time_us":203711,"thread_start_us":187373,"threads_started":2}]]}
I20260709 12:16:17.237628 23395 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:17.250440 23287 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:17.256023 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:17.259145 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 164393us
I20260709 12:16:17.269924 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:17.270803 24013 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:17.227361 30691 raft_consensus.cc:1081] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:17.307598 23214 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 17 to 18, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.257678 30532 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:17.345448 23469 raft_consensus.cc:1240] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Rejecting Update request from peer 54640ab8abeb4c949426d66c7e5aed87 for earlier term 12. Current term is 13. Ops: []
I20260709 12:16:17.359668 24012 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:17.358049 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:17.373929 24010 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:16:17.374279 24010 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:17.390415 23388 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:17.366454 30532 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.399044 23214 catalog_manager.cc:5697] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 18 to 22, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 22 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:17.407805 23214 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 11 to 13, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:17.408929 23319 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:17.413020 23314 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:17.413718 23314 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:17.421526 23314 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:17.426277 24013 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 16.
I20260709 12:16:17.401000 24010 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.437825 24010 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.442234 23312 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:17.446457 30804 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:16:17.413952 30532 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.453733 23312 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:17.456607 24010 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:17.463018 24012 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 12:16:17.468417 30799 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.075s	user 0.000s	sys 0.003s
I20260709 12:16:17.489992 23314 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.498473 23316 raft_consensus.cc:1240] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 14. Current term is 15. Ops: []
W20260709 12:16:17.493816 30807 log.cc:927] Time spent T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.055s	user 0.002s	sys 0.000s
I20260709 12:16:17.512506 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:17.513303 23316 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:17.513809 23316 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:17.540499 24010 raft_consensus.cc:1240] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 22. Current term is 23. Ops: []
I20260709 12:16:17.540606 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:17.542382 23468 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.536132 23314 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:17.540750 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:17.546640 24012 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:16:17.547045 24012 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:17.553653 30801 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:17.554569 30801 consensus_queue.cc:1243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:16:17.555051 30715 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:16:17.557358 23314 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:17.560736 30715 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:17.558123 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:17.560137 30774 consensus_queue.cc:1059] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:17.561967 23389 raft_consensus.cc:3055] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:16:17.562321 23389 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:17.562426 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:17.563016 23466 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:17.563196 23389 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.563558 30798 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:17.564632 23389 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:17.565161 23287 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:17.565701 24012 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.566637 30503 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:17.567221 24012 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:16:17.567037 30798 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:17.567499 30503 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.569094 30503 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.576786 24012 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:17.584885 23468 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 13.
I20260709 12:16:17.588553 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:17.589710 30796 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:16:17.590258 30796 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.591212 30796 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.591643 30766 consensus_queue.cc:1059] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:17.592628 30776 raft_consensus.cc:3055] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:17.592998 30776 raft_consensus.cc:740] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:17.593864 30776 consensus_queue.cc:260] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.595459 30776 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:17.602592 24010 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:16:17.604713 30276 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:17.606678 23312 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:16:17.583534 23316 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.650934 23468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:17.652506 23468 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:17.652629 23389 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:17.651683 23316 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:17.565085 30715 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.656844 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:17.662781 24012 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:16:17.663275 24012 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:17.659140 23287 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:17.669725 23391 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:17.669586 24012 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.679432 24012 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:16:17.648149 30624 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 12:16:17.660498 30774 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:16:17.682264 23313 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:17.682071 30532 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:17.700953 30532 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.712782 30532 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:17.701097 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 70812.3us
I20260709 12:16:17.701673 30504 consensus_queue.cc:1059] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:16:17.709631 30673 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:16:17.710014 30817 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
I20260709 12:16:17.711778 23465 raft_consensus.cc:1275] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:16:17.715195 24012 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:16:17.741034 24010 raft_consensus.cc:1240] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 15. Current term is 16. Ops: []
I20260709 12:16:17.743957 23468 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 10.
I20260709 12:16:17.753459 24010 raft_consensus.cc:1275] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:17.743705 30624 consensus_queue.cc:1243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer 2778b449761a46708f2f643b1b5b5854 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:16:17.770133 30812 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:16:17.781867 30820 raft_consensus.cc:3055] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:17.782315 30820 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:17.744379 30438 consensus_queue.cc:1048] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:16:17.799129 30715 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:16:17.737862 30767 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.026s
W20260709 12:16:17.841861 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 156876us
I20260709 12:16:17.846832 23466 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 15.
I20260709 12:16:17.845057 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:17.858734 23465 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:16:17.863750 23388 raft_consensus.cc:1240] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Rejecting Update request from peer da1e2fd157894634bf92d6c82c06ec64 for earlier term 16. Current term is 17. Ops: []
I20260709 12:16:17.862534 30783 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:16:17.881615 30795 consensus_queue.cc:1048] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:17.884078 30829 consensus_queue.cc:1059] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:16:17.891518 23389 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:16:17.894780 23389 pending_rounds.cc:85] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting all ops after (but not including) 12
I20260709 12:16:17.895061 23389 pending_rounds.cc:107] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:16:17.895403 23389 raft_consensus.cc:2889] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:17.908316 30742 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:16:17.901764 30783 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:17.925688 30783 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:17.927160 30820 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:17.933476 24013 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:16:17.944702 23389 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:17.931706 30503 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:16:17.954111 30503 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:17.981390 30820 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:18.015357 23462 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:16:18.023377 30783 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:18.061785 23469 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:16:18.062261 23469 pending_rounds.cc:85] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 9
I20260709 12:16:18.062480 23469 pending_rounds.cc:107] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:16:18.062712 23469 raft_consensus.cc:2889] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:18.082455 30783 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:16:18.119693 24007 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:16:18.125296 24010 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:16:18.137902 23362 consensus_peers.cc:597] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa5f71c212664b78a4deb373bf5ed2f5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.153636 30792 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:18.155323 23388 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:16:18.155587 30792 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:18.159786 23469 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:18.161037 23462 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:16:18.161523 23462 pending_rounds.cc:85] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Aborting all ops after (but not including) 12
I20260709 12:16:18.161808 23462 pending_rounds.cc:107] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:16:18.161643 30276 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:16:18.162231 23462 raft_consensus.cc:2889] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:18.185180 23391 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:16:18.192225 30792 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:16:18.194161 30715 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:16:18.198592 30784 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:16:18.236081 23461 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:16:18.243094 30792 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:18.264407 23312 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:18.284322 23217 catalog_manager.cc:5697] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.279187 30504 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:16:18.345218 23470 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:16:18.348250 30796 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:18.396199 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:18.416160 23217 catalog_manager.cc:5697] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 21 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.417749 23217 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.419235 23217 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 16 to 18, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 18 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.420605 23217 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 19 to 21, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 21 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.421988 23217 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 11, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 11 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.423530 23217 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 21 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.427579 23217 catalog_manager.cc:5697] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 17 to 22, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 22 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.439047 24007 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:16:18.440560 30792 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:18.443897 30806 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.447358 23461 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.456243 23214 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 321) took 9988 ms. Trace:
I20260709 12:16:18.460651 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:18.474284 23217 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 21 to 23, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 23 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.476594 23217 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.483349 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet ec4e3b5eed804f2a8c2a01e5d1ae5f53 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:18.456710 23214 rpcz_store.cc:276] 0709 12:16:08.467848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:08.468196 (+   348us) service_pool.cc:224] Handling call
0709 12:16:18.456151 (+9987955us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:17.587072 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:17.587810 (+   738us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:17.587862 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:16:17.587938 (+    76us) tablet.cc:662] Decoding operations
0709 12:16:17.589670 (+  1732us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:17.589784 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:17.589811 (+    27us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:16:17.591008 (+  1197us) tablet.cc:701] Row locks acquired
0709 12:16:17.591077 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:16:17.591337 (+   260us) write_op.cc:270] Start()
0709 12:16:17.591551 (+   214us) write_op.cc:276] Timestamp: P: 1783599377591237 usec, L: 0
0709 12:16:17.591577 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:16:17.592504 (+   927us) log.cc:844] Serialized 3673 byte log entry
0709 12:16:17.762021 (+169517us) op_driver.cc:464] REPLICATION: finished
0709 12:16:17.834402 (+ 72381us) write_op.cc:301] APPLY: starting
0709 12:16:17.834487 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:17.973258 (+138771us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b180040d140. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af45c281 00007ff0a93b30ec 00007ff0a93ffc0a 00007ff0a93ff4f7 00007ff0a93ff490 00007ff0a93aa754 00007ff0ad0caf13 00007ff0ad0cb532 00007ff0ad112b93 00007ff0acfbfde3
0709 12:16:18.024949 (+ 51691us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:18.024974 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:18.067370 (+ 42396us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:18.067628 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:16:18.067678 (+    50us) write_op.cc:312] APPLY: finished
0709 12:16:18.067968 (+   290us) log.cc:844] Serialized 89 byte log entry
0709 12:16:18.121525 (+ 53557us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:18.179218 (+ 57693us) write_op.cc:454] Released schema lock
0709 12:16:18.189802 (+ 10584us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:18.203155 (+ 13353us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:18.204018 (+   863us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":100536,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":72127,"apply.run_cpu_time_us":19068,"apply.run_wall_time_us":363003,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41783,"num_ops":9,"prepare.queue_time_us":16537,"prepare.run_cpu_time_us":43512,"prepare.run_wall_time_us":66387,"raft.queue_time_us":71336,"raft.run_cpu_time_us":16908,"raft.run_wall_time_us":85795,"replication_time_us":170238,"spinlock_wait_cycles":2623232,"thread_start_us":159076,"threads_started":4,"wal-append.queue_time_us":95634}]]}
I20260709 12:16:18.499150 23217 catalog_manager.cc:5697] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:18.501233 23217 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 22 to 23, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 23 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.502980 23217 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 19 to 20, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 20 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.504712 23217 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 19 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 20 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:18.507179 23316 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.506135 23217 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 13 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:18.534236 23267 auto_rebalancer.cc:629] Failed to schedule move for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29: Illegal state: Replica 6253b834ffb744579b46fcede2fc0e91 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:18.540025 23388 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:18.564340 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:18.564993 24012 raft_consensus.cc:606] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:18.570446 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:18.571095 23391 raft_consensus.cc:606] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:18.579137 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "446014759a9c4a0f9c99759951367083"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:18.579707 24012 raft_consensus.cc:606] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:18.586136 24007 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:18.586714 24007 raft_consensus.cc:606] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:18.594173 24012 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:18.602017 24012 raft_consensus.cc:606] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:18.602543 24012 raft_consensus.cc:614] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:18.605356 24013 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:41504
I20260709 12:16:18.606005 24013 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:18.606371 24013 raft_consensus.cc:614] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:18.607378 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:16:18.609278 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:18.610092 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:18.621322 23461 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:16:18.622269 23362 consensus_peers.cc:597] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2984a5ac8b6b49c9aa8eb969a484d610. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.624696 30673 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:16:18.658608 30673 raft_consensus.cc:993] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:18.659142 30673 raft_consensus.cc:1081] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:18.660904 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:18.661535 23316 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.661921 23316 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:18.674358 30797 raft_consensus.cc:993] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:18.674969 30797 raft_consensus.cc:1081] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:18.679770 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:60318
I20260709 12:16:18.680328 23462 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.680691 23462 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:18.682258 23319 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:16:18.690443 30260 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:18.695138 23462 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:18.695330 23316 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:18.700024 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:18.700906 23312 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:18.704962 23462 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:18.706166 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:18.706753 24013 raft_consensus.cc:3055] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:18.707041 24013 raft_consensus.cc:740] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:18.709962 23316 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:18.707772 24013 consensus_queue.cc:260] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:18.710844 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:18.711035 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:18.711642 23388 raft_consensus.cc:3055] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:18.712103 23388 raft_consensus.cc:740] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:18.711794 23461 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:18.701937 30504 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.718976 23470 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.716943 23388 consensus_queue.cc:260] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:18.741917 23388 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:18.740116 24013 raft_consensus.cc:3060] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:18.733707 23312 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:18.763190 23434 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:18.764256 30795 raft_consensus.cc:2804] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:18.774075 24013 raft_consensus.cc:2468] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:18.764742 30795 raft_consensus.cc:697] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:18.800029 30795 consensus_queue.cc:237] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:18.802057 23388 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:18.806315 23287 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:18.814997 30796 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:18.828872 23461 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 21.
I20260709 12:16:18.831748 30796 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:18.832711 30796 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:18.833521 23388 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:18.838860 30861 ts_tablet_manager.cc:933] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:18.862910 23319 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:18.898217 30727 raft_consensus.cc:993] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:18.898904 30808 raft_consensus.cc:1081] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:18.902486 23461 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:60318
I20260709 12:16:18.903419 23461 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.903779 23461 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:18.905282 23470 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:16:18.905700 23362 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfaa0d3f687341a1ac860baa49e8bfb4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:18.910506 23316 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:16:18.912603 30260 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:18.916091 30861 tablet_copy_client.cc:323] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:16:18.929575 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet fa5f71c212664b78a4deb373bf5ed2f5 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:16:18.930269 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet fa5f71c212664b78a4deb373bf5ed2f5 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-fa5f71c212664b78a4deb373bf5ed2f5
I20260709 12:16:18.934556 30260 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:18.956967 30697 raft_consensus.cc:993] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:18.958168 30697 raft_consensus.cc:1081] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:18.960789 23313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "446014759a9c4a0f9c99759951367083"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:18.961508 23313 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:18.961916 23313 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:19.011178 23215 catalog_manager.cc:5697] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.013298 23215 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.015295 23215 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 16 to 21, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.017550 23406 tablet_copy_source_session.cc:215] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:19.017037 23215 catalog_manager.cc:5697] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 20 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 21 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.021973 23461 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:19.031096 23215 catalog_manager.cc:5697] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 20 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.033413 23461 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:19.033308 23319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:19.033335 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:19.033207 23215 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.034171 23319 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:19.034438 24013 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:16:19.034772 24013 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:19.035531 24013 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:19.035143 23215 catalog_manager.cc:5697] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 15, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 15 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.036652 24013 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:19.044021 30861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa5f71c212664b78a4deb373bf5ed2f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:19.044231 23215 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.061354 24013 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:19.064647 23319 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
I20260709 12:16:19.029011 23313 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:19.067206 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:19.083535 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:19.084479 23388 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:19.087227 23313 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:19.104704 30795 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:19.132562 30260 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:19.150307 23388 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:19.142699 30795 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:19.161985 30869 ts_tablet_manager.cc:933] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:19.182734 24007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:19.183422 24007 raft_consensus.cc:3055] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:19.183698 24007 raft_consensus.cc:740] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:19.183990 23287 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:19.195712 24007 consensus_queue.cc:260] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:19.201613 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 121848us
I20260709 12:16:19.202644 24007 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:19.191617 30276 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:19.234540 30869 tablet_copy_client.cc:323] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:16:19.245038 24007 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 18.
I20260709 12:16:19.229750 30276 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:19.227785 30795 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:19.257126 23466 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:19.266350 23313 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:19.270082 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 2984a5ac8b6b49c9aa8eb969a484d610 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:16:19.270689 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 2984a5ac8b6b49c9aa8eb969a484d610 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-2984a5ac8b6b49c9aa8eb969a484d610
I20260709 12:16:19.304913 30276 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:19.309484 23406 tablet_copy_source_session.cc:215] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:19.339330 30869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2984a5ac8b6b49c9aa8eb969a484d610. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:19.344714 23211 catalog_manager.cc:5697] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 21 to 22, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 22 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.347100 23211 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 14 to 16, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 16 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.348809 23211 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 12 to 13, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.350528 23211 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 21 to 23, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 23 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.367035 23313 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:19.368712 30795 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:19.352205 23211 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 12 to 13, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 13 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.375356 23215 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 348) took 3802 ms. Trace:
I20260709 12:16:19.376067 23211 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 9 to 10, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 10 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.377230 23215 rpcz_store.cc:276] 0709 12:16:15.573015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:15.577084 (+  4069us) service_pool.cc:224] Handling call
0709 12:16:19.375250 (+3798166us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:19.062876 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:19.063627 (+   751us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:19.063669 (+    42us) write_op.cc:435] Acquired schema lock
0709 12:16:19.063686 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:19.072907 (+  9221us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:19.073004 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:19.073045 (+    41us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:16:19.073971 (+   926us) tablet.cc:701] Row locks acquired
0709 12:16:19.073997 (+    26us) write_op.cc:260] PREPARE: finished
0709 12:16:19.074206 (+   209us) write_op.cc:270] Start()
0709 12:16:19.074358 (+   152us) write_op.cc:276] Timestamp: P: 1783599379074140 usec, L: 0
0709 12:16:19.074387 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 12:16:19.075263 (+   876us) log.cc:844] Serialized 3277 byte log entry
0709 12:16:19.187003 (+111740us) op_driver.cc:464] REPLICATION: finished
0709 12:16:19.197252 (+ 10249us) write_op.cc:301] APPLY: starting
0709 12:16:19.199964 (+  2712us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:19.215724 (+ 15760us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:19.215838 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:19.310675 (+ 94837us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:19.310902 (+   227us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:16:19.310946 (+    44us) write_op.cc:312] APPLY: finished
0709 12:16:19.311198 (+   252us) log.cc:844] Serialized 81 byte log entry
0709 12:16:19.314068 (+  2870us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.314179 (+   111us) write_op.cc:454] Released schema lock
0709 12:16:19.314227 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:19.314679 (+   452us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.314714 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":4381,"apply.run_cpu_time_us":12909,"apply.run_wall_time_us":119904,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33472,"num_ops":8,"prepare.queue_time_us":9367,"prepare.run_cpu_time_us":4767,"prepare.run_wall_time_us":12692,"replication_time_us":112433}]]}
I20260709 12:16:19.412436 23388 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:16:19.415057 30792 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:19.419126 24012 raft_consensus.cc:1275] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:19.420554 30795 consensus_queue.cc:1048] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:19.430179 23470 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:16:19.432322 30784 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:19.443100 30861 tablet_copy_client.cc:806] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:19.468258 30883 ts_tablet_manager.cc:933] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:19.479329 30861 tablet_copy_client.cc:670] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:19.537158 30869 tablet_copy_client.cc:806] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:19.538305 30869 tablet_copy_client.cc:670] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:19.542259 30861 tablet_copy_client.cc:538] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:19.581075 30883 tablet_copy_client.cc:323] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:16:19.599895 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet cfaa0d3f687341a1ac860baa49e8bfb4 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:16:19.600524 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet cfaa0d3f687341a1ac860baa49e8bfb4 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-cfaa0d3f687341a1ac860baa49e8bfb4
I20260709 12:16:19.628342 30869 tablet_copy_client.cc:538] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:19.636956 23406 tablet_copy_source_session.cc:215] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:19.651461 30883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cfaa0d3f687341a1ac860baa49e8bfb4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:19.658068 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 153940us
I20260709 12:16:19.667241 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.654419 23212 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 19 to 21, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 21 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.702929 30869 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:19.707516 23313 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:16:19.709340 30767 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:19.715358 23211 catalog_manager.cc:5697] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 17 to 18, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.753921 30861 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:19.766772 30883 tablet_copy_client.cc:806] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:19.767436 30883 tablet_copy_client.cc:670] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:19.769973 24012 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:16:19.774235 30883 tablet_copy_client.cc:538] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:19.788257 30820 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:19.833235 23388 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:19.841233 23214 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.847343 23214 catalog_manager.cc:5697] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:19.850749 23214 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 15 to 17, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.852228 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 14 to 15, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 15 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.858132 23388 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:19.858754 23388 raft_consensus.cc:606] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:19.845841 30880 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:16:19.887588 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:19.888191 23391 raft_consensus.cc:606] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:19.888527 23391 raft_consensus.cc:614] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:19.892318 24013 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:19.900323 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:19.903034 23391 raft_consensus.cc:606] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:19.908437 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:19.895401 30792 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 12:16:19.909139 23395 raft_consensus.cc:606] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:19.911535 23211 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 16, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 16 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:19.913893 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:19.914520 23395 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:19.915745 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:19.917968 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:19.896507 23214 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:19.925354 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:19.947958 30883 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:19.964799 23214 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:19.997592 23212 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 332) took 4808 ms. Trace:
I20260709 12:16:20.002395 23212 rpcz_store.cc:276] 0709 12:16:15.188710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:15.188939 (+   229us) service_pool.cc:224] Handling call
0709 12:16:19.997503 (+4808564us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:19.699977 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:19.700497 (+   520us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:19.700530 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:16:19.700545 (+    15us) tablet.cc:662] Decoding operations
0709 12:16:19.700869 (+   324us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:19.700918 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:19.700944 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:19.701126 (+   182us) tablet.cc:701] Row locks acquired
0709 12:16:19.701142 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:16:19.701279 (+   137us) write_op.cc:270] Start()
0709 12:16:19.701384 (+   105us) write_op.cc:276] Timestamp: P: 1783599379701235 usec, L: 0
0709 12:16:19.701403 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:16:19.701997 (+   594us) log.cc:844] Serialized 544 byte log entry
0709 12:16:19.703993 (+  1996us) op_driver.cc:464] REPLICATION: finished
0709 12:16:19.704440 (+   447us) write_op.cc:301] APPLY: starting
0709 12:16:19.704517 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:19.705305 (+   788us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:19.705398 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:19.707506 (+  2108us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:19.707650 (+   144us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:16:19.707694 (+    44us) write_op.cc:312] APPLY: finished
0709 12:16:19.707891 (+   197us) log.cc:844] Serialized 25 byte log entry
0709 12:16:19.708065 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.708143 (+    78us) write_op.cc:454] Released schema lock
0709 12:16:19.708180 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:19.708546 (+   366us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:19.708573 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":62295,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":238,"apply.run_cpu_time_us":2736,"apply.run_wall_time_us":4060,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":2291,"prepare.run_wall_time_us":2291,"replication_time_us":2468}]]}
I20260709 12:16:20.116904 30673 raft_consensus.cc:993] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:20.117470 30673 raft_consensus.cc:1081] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:20.119553 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:16:20.120098 23466 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.120419 23466 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:20.149545 23211 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 17 to 18, leader changed from 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 18 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:20.181551 30869 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:20.183368 30869 tablet_bootstrap.cc:492] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:20.184430 30869 ts_tablet_manager.cc:1403] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.482s	user 0.038s	sys 0.018s
I20260709 12:16:20.169899 23466 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.199009 23215 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 13, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:20.205361 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:20.206174 23388 raft_consensus.cc:3055] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:16:20.206496 23388 raft_consensus.cc:740] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:20.207296 23388 consensus_queue.cc:260] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.217028 30874 raft_consensus.cc:993] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:20.229700 30874 raft_consensus.cc:1081] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:20.236546 23313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:20.237162 23313 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.237563 23313 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:20.210696 23388 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:20.245661 23466 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:20.247200 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19d9facb5378417aaa3279b3dd9f02a9" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:20.247798 24013 raft_consensus.cc:3060] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:20.251134 30869 raft_consensus.cc:359] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.252199 30869 raft_consensus.cc:740] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:20.228894 30861 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:20.254906 30861 tablet_bootstrap.cc:492] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:20.255677 30861 ts_tablet_manager.cc:1403] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.504s	user 0.039s	sys 0.011s
I20260709 12:16:20.276542 24013 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:20.278934 23436 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:20.279824 30879 raft_consensus.cc:2804] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:20.280316 30879 raft_consensus.cc:697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:20.281152 30879 consensus_queue.cc:237] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.290002 30861 raft_consensus.cc:359] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.291052 30861 raft_consensus.cc:740] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:20.291813 30861 consensus_queue.cc:260] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.252965 30869 consensus_queue.cc:260] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.297101 30869 ts_tablet_manager.cc:1434] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.112s	user 0.003s	sys 0.006s
I20260709 12:16:20.302009 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-2984a5ac8b6b49c9aa8eb969a484d610 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:16:20.302477 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-2984a5ac8b6b49c9aa8eb969a484d610 on tablet 2984a5ac8b6b49c9aa8eb969a484d610 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:20.313066 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:20.313796 23391 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Received request to transfer leadership
I20260709 12:16:20.315706 23214 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 322) took 1473 ms. Trace:
I20260709 12:16:20.326006 30861 ts_tablet_manager.cc:1434] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.003s
I20260709 12:16:20.337838 23391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:20.338702 23391 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Received request to transfer leadership
W20260709 12:16:20.340159 23206 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for cfaa0d3f687341a1ac860baa49e8bfb4 already in progress
I20260709 12:16:20.347990 23313 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.352672 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-fa5f71c212664b78a4deb373bf5ed2f5 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:16:20.318243 23214 rpcz_store.cc:276] 0709 12:16:18.842234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:18.842415 (+   181us) service_pool.cc:224] Handling call
0709 12:16:18.919726 (+ 77311us) spinlock_profiling.cc:243] Waited 62 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:20.315623 (+1395897us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:20.033479 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:20.034060 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:20.034097 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:16:20.034114 (+    17us) tablet.cc:662] Decoding operations
0709 12:16:20.034849 (+   735us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:20.034898 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:20.034926 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:16:20.035450 (+   524us) tablet.cc:701] Row locks acquired
0709 12:16:20.035467 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:16:20.035590 (+   123us) write_op.cc:270] Start()
0709 12:16:20.035701 (+   111us) write_op.cc:276] Timestamp: P: 1783599380035548 usec, L: 0
0709 12:16:20.035718 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:16:20.036492 (+   774us) log.cc:844] Serialized 2618 byte log entry
0709 12:16:20.042216 (+  5724us) op_driver.cc:464] REPLICATION: finished
0709 12:16:20.045765 (+  3549us) write_op.cc:301] APPLY: starting
0709 12:16:20.045857 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:20.061816 (+ 15959us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:20.061961 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:20.092286 (+ 30325us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:20.092555 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 12:16:20.092595 (+    40us) write_op.cc:312] APPLY: finished
0709 12:16:20.125548 (+ 32953us) log.cc:844] Serialized 65 byte log entry
0709 12:16:20.126293 (+   745us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:20.126413 (+   120us) write_op.cc:454] Released schema lock
0709 12:16:20.126477 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:20.136953 (+ 10476us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:20.136994 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":217257,"reported_tablets":36,"spinlock_wait_cycles":142691712,"child_traces":[["op",{"apply.queue_time_us":629,"apply.run_cpu_time_us":10766,"apply.run_wall_time_us":91185,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25104,"num_ops":6,"prepare.queue_time_us":243,"prepare.run_cpu_time_us":3269,"prepare.run_wall_time_us":8228,"replication_time_us":6378}]]}
I20260709 12:16:20.364565 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-fa5f71c212664b78a4deb373bf5ed2f5 on tablet fa5f71c212664b78a4deb373bf5ed2f5 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:20.367352 23313 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 24 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:20.368357 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:20.368213 23388 raft_consensus.cc:2468] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 22.
I20260709 12:16:20.368909 24013 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:20.368947 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:20.369709 23391 raft_consensus.cc:3055] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:20.370034 23391 raft_consensus.cc:740] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:20.370756 23391 consensus_queue.cc:260] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.402707 24013 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 24.
I20260709 12:16:20.404848 23391 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:20.408795 23389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:20.411060 23389 raft_consensus.cc:606] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Received request to transfer leadership
I20260709 12:16:20.420238 23287 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:20.442636 23391 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 24.
W20260709 12:16:20.445688 23267 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
I20260709 12:16:20.445855 30673 raft_consensus.cc:993] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:16:20.446317 23267 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for cfaa0d3f687341a1ac860baa49e8bfb4 already in progress
I20260709 12:16:20.446450 30673 raft_consensus.cc:1081] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
W20260709 12:16:20.446801 23267 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for cfaa0d3f687341a1ac860baa49e8bfb4 already in progress
I20260709 12:16:20.429064 30870 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:20.449927 30877 raft_consensus.cc:993] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:20.450496 30877 raft_consensus.cc:1081] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:20.449883 30870 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 24 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:20.452065 30870 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.463887 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:16:20.464522 23470 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.464847 23470 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:20.477707 23313 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:20.478374 23313 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.478749 23313 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:20.493402 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 65994.5us
I20260709 12:16:20.500031 23313 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.517557 23313 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:20.520761 30873 raft_consensus.cc:993] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:20.521332 30873 raft_consensus.cc:1081] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:20.521553 30883 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:20.522567 30883 tablet_bootstrap.cc:492] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:20.523310 30883 ts_tablet_manager.cc:1403] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.584s	user 0.077s	sys 0.067s
I20260709 12:16:20.523449 23321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:20.524084 23321 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:20.524451 23321 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:20.510658 23470 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:20.535410 23216 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 331) took 23503 ms (client timeout 15000 ms). Trace:
W20260709 12:16:20.540796 23216 rpcz_store.cc:269] 0709 12:15:57.031942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:57.032150 (+   208us) service_pool.cc:224] Handling call
0709 12:16:20.535202 (+23503052us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54,"reported_tablets":46}
I20260709 12:16:20.538391 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:20.540268 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:20.543041 23466 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:20.543112 23393 raft_consensus.cc:3055] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:16:20.543576 23393 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:20.544627 23470 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:20.544919 23393 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.547050 23393 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:20.549292 23321 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.551254 23321 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:20.553246 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:20.554000 23391 raft_consensus.cc:3055] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:20.554291 23391 raft_consensus.cc:740] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:20.555277 23391 consensus_queue.cc:260] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:20.555533 23393 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:16:20.562251 23393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:20.563117 23393 raft_consensus.cc:3055] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:20.563545 23393 raft_consensus.cc:740] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:20.564678 23391 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:20.561494 23287 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:20.566149 23466 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 19.
I20260709 12:16:20.567386 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:20.568148 23321 raft_consensus.cc:3060] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:20.572367 23393 consensus_queue.cc:260] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.574515 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:20.575313 24013 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:20.586064 23393 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:20.569356 30906 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:20.597951 23393 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:20.604132 24013 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:20.595777 23391 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:20.605014 23287 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:20.614472 23321 raft_consensus.cc:2468] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:16:20.648649 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43719.8us
W20260709 12:16:20.650760 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44578.5us
I20260709 12:16:20.650234 23434 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:20.614202 30906 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:20.654168 30784 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:16:20.695201 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 90979.9us
W20260709 12:16:20.704108 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 53948.8us
I20260709 12:16:20.683213 30905 raft_consensus.cc:2804] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:20.689509 30906 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:20.686362 30784 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:20.715087 30784 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:20.724072 30905 raft_consensus.cc:697] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:20.784288 23212 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 20 to 21, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 21 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } }
I20260709 12:16:20.800100 24007 raft_consensus.cc:1217] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 12:16:20.856150 30905 consensus_queue.cc:237] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:20.937852 24007 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:16:20.985512 30883 raft_consensus.cc:359] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:21.038681 24007 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:16:21.050069 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 74207.8us
I20260709 12:16:21.036983 30883 raft_consensus.cc:740] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:21.064843 30919 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.009s
I20260709 12:16:21.079805 30883 consensus_queue.cc:260] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:21.083285 30883 ts_tablet_manager.cc:1434] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.526s	user 0.006s	sys 0.008s
W20260709 12:16:21.083738 30883 ts_tablet_manager.cc:1467] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Tablet startup took 526ms
W20260709 12:16:21.084100 30883 ts_tablet_manager.cc:1469] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Trace:
0709 12:16:19.341086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:19.341266 (+   180us) service_pool.cc:224] Handling call
0709 12:16:19.468654 (+127388us) ts_tablet_manager.cc:934] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
0709 12:16:19.650172 (+181518us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:16:19.756920 (+106748us) tablet_metadata.cc:665] Metadata flushed
0709 12:16:19.757049 (+   129us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:16:19.765298 (+  8249us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:16:19.765657 (+   359us) tablet_replica.cc:206] Creating consensus instance
0709 12:16:19.765983 (+   326us) ts_tablet_manager.cc:989] Replica registered.
0709 12:16:19.766044 (+    61us) inbound_call.cc:177] Queueing success response
0709 12:16:19.938705 (+172661us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:16:20.557196 (+618491us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:16:20.717719 (+160523us) tablet_replica.cc:261] Starting instrumentation
0709 12:16:20.718537 (+   818us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":33096,"mutex_wait_us":7732,"raft.queue_time_us":1975,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"tablet-copy.queue_time_us":116206,"tablet-download-pool.queue_time_us":124,"tablet-download-pool.run_cpu_time_us":1159,"tablet-download-pool.run_wall_time_us":6106,"thread_start_us":177070,"threads_started":2}
I20260709 12:16:21.112114 30904 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:16:21.121133 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 149762us
I20260709 12:16:21.147028 23211 catalog_manager.cc:5697] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 21 to 22, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 22 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:21.155419 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 113020us
I20260709 12:16:21.164475 24007 raft_consensus.cc:1217] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.12->[17.13-17.13]   Dedup: 17.13->[]
I20260709 12:16:21.183023 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-cfaa0d3f687341a1ac860baa49e8bfb4 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:16:21.183508 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-cfaa0d3f687341a1ac860baa49e8bfb4 on tablet cfaa0d3f687341a1ac860baa49e8bfb4 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:21.218138 30938 raft_consensus.cc:1064] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:16:21.221339 30938 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:21.187009 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:21.261878 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 131501us
I20260709 12:16:21.293123 24012 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 LEARNER]: Advancing to term 19
I20260709 12:16:21.306977 24012 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:16:21.309405 23395 raft_consensus.cc:1275] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:16:21.316954 23391 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:21.322196 24007 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:16:21.327584 23389 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:16:21.336473 23461 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:21.340391 23466 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:16:21.349620 24007 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:21.354096 30820 consensus_queue.cc:1048] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:21.358160 30894 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:21.359716 30906 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:21.351254 30938 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:16:21.362859 30876 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:21.356674 23218 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 330) took 43741 ms (client timeout 15000 ms). Trace:
W20260709 12:16:21.365130 23218 rpcz_store.cc:269] 0709 12:15:37.615048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:37.630663 (+ 15615us) service_pool.cc:224] Handling call
0709 12:16:21.356467 (+43725804us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":943198,"reported_tablets":38}
I20260709 12:16:21.366604 30919 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:16:21.374028 23313 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:21.417192 30919 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:16:21.419689 23395 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:16:21.422490 23395 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:16:21.431407 30877 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:21.439845 23466 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:21.442888 23214 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 333) took 1070 ms. Trace:
I20260709 12:16:21.443279 23214 rpcz_store.cc:276] 0709 12:16:20.372051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:20.372175 (+   124us) service_pool.cc:224] Handling call
0709 12:16:21.442819 (+1070644us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124989,"reported_tablets":27}
I20260709 12:16:21.475136 30673 raft_consensus.cc:1064] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:16:21.481626 24012 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:21.482723 30673 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:21.504958 23313 raft_consensus.cc:1275] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:16:21.505872 23214 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 18 to 19, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 19 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:21.498224 30906 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:16:21.503618 30876 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:16:21.537458 23214 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 23 to 24, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 24 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:21.616873 23395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:21.617573 23395 raft_consensus.cc:606] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:21.578066 30276 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:16:21.573978 30940 consensus_queue.cc:1048] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:16:21.579471 30908 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 12:16:21.675788 30951 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:16:21.690650 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 116217us
I20260709 12:16:21.788842 30910 raft_consensus.cc:1064] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:16:21.793601 30910 raft_consensus.cc:1068] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: Unable to promote non-voter 54640ab8abeb4c949426d66c7e5aed87: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:16:21.819864 23313 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:21.861048 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 115133us
I20260709 12:16:21.880779 24015 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:16:21.885998 30920 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:16:22.034176 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 87246.6us
I20260709 12:16:22.045921 23462 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:16:22.105870 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 58801.7us
W20260709 12:16:22.114419 30957 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:22.179939 24009 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
W20260709 12:16:22.198096 30973 log.cc:927] Time spent T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:16:22.267410 30981 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:22.272987 23214 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 19 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 20 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:22.276516 23313 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:22.277303 23313 raft_consensus.cc:606] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:22.287984 23313 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:22.288715 23313 raft_consensus.cc:606] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:22.313761 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:22.314440 23392 raft_consensus.cc:606] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:22.345928 30792 raft_consensus.cc:1064] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:16:22.350104 30967 raft_consensus.cc:993] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:22.350651 30967 raft_consensus.cc:1081] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:22.348665 30792 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.373850 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:16:22.374495 23462 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:22.374861 23462 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:22.382349 23396 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:22.383078 23396 raft_consensus.cc:606] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:22.383399 23396 raft_consensus.cc:614] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:22.384611 23462 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.387382 23462 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:22.390354 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:16:22.392223 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:22.392659 23396 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
W20260709 12:16:22.394536 23219 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 346) took 36039 ms (client timeout 15000 ms). Trace:
W20260709 12:16:22.394985 23219 rpcz_store.cc:269] 0709 12:15:46.354713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:46.365973 (+ 11260us) service_pool.cc:224] Handling call
0709 12:16:22.394331 (+36028358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":270183,"reported_tablets":48}
I20260709 12:16:22.397115 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:22.403105 23321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:22.403733 23321 raft_consensus.cc:3055] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:16:22.404053 23321 raft_consensus.cc:740] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:22.410013 23321 consensus_queue.cc:260] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.413843 23321 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:22.462838 23321 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:16:22.481359 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:22.482298 24011 raft_consensus.cc:3060] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:16:22.484916 30909 raft_consensus.cc:993] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:22.485476 30909 raft_consensus.cc:1081] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:22.469722 30914 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:22.490208 30914 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:22.492359 30914 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:22.465200 30958 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.035s
I20260709 12:16:22.501142 24011 raft_consensus.cc:2468] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:16:22.504367 23470 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:16:22.504987 23470 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:22.505343 23470 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:22.518227 24011 raft_consensus.cc:1240] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 10. Current term is 11. Ops: []
I20260709 12:16:22.523742 23434 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:22.538443 30965 raft_consensus.cc:2804] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:16:22.539009 30965 raft_consensus.cc:697] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:22.540544 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:22.560842 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:22.559680 30993 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:16:22.578513 30919 raft_consensus.cc:993] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:16:22.582232 23470 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:22.583657 23461 raft_consensus.cc:1240] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 19. Current term is 20. Ops: []
I20260709 12:16:22.587050 23470 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:22.539835 30965 consensus_queue.cc:237] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.596343 30997 consensus_queue.cc:1059] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:22.597457 30997 raft_consensus.cc:3055] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:22.597888 30997 raft_consensus.cc:740] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:22.598737 30997 consensus_queue.cc:260] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:22.599882 30997 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:22.597477 30937 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:22.610972 23462 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:22.595713 30919 raft_consensus.cc:1081] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:16:22.625952 23470 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:22.628669 24009 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:22.631945 23392 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:16:22.633780 23360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.008s
W20260709 12:16:22.634132 23360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.008s	sys 0.008s
W20260709 12:16:22.634441 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 551669us
I20260709 12:16:22.652571 23316 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:16:22.662768 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:22.663350 23316 raft_consensus.cc:3060] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:22.675103 23287 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:22.678027 24009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:16:22.678818 24009 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:22.683007 24009 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:22.678750 24011 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:22.693184 30997 raft_consensus.cc:993] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:22.693761 30997 raft_consensus.cc:1081] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
W20260709 12:16:22.682785 31003 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
I20260709 12:16:22.705343 23316 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:16:22.684859 30914 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:16:22.726948 30914 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:22.730332 30792 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:22.731408 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:22.733201 23434 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:22.741328 23461 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:16:22.742026 23461 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:22.743510 23461 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:22.747932 23396 raft_consensus.cc:2468] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:16:22.752454 24009 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.728637 23312 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:22.760896 30982 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:22.757231 30905 raft_consensus.cc:2804] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:22.786675 24009 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:22.790753 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:22.793098 23312 raft_consensus.cc:3055] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:22.793421 23312 raft_consensus.cc:740] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:22.795886 23312 consensus_queue.cc:260] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.797056 23312 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:22.806559 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:22.807278 23392 raft_consensus.cc:3060] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:22.819410 23312 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 24.
I20260709 12:16:22.788214 30989 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:22.829680 23980 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:22.836671 23461 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:22.807616 30905 raft_consensus.cc:697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:22.841203 30727 raft_consensus.cc:2804] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:22.859179 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:22.859807 23312 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:22.860316 23461 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:22.871104 23312 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:22.873019 23434 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:22.860443 30993 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:22.864952 30727 raft_consensus.cc:697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:22.884650 30727 consensus_queue.cc:237] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:22.912544 31010 raft_consensus.cc:2804] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:22.881733 30993 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:22.881309 30905 consensus_queue.cc:237] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:22.992457 31005 log.cc:927] Time spent T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 12:16:22.857834 30958 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Committing config change with OpId 23.19: config changed from index 7 to 19, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.049692 23392 raft_consensus.cc:2468] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 24.
I20260709 12:16:23.088330 23316 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.114651 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:23.115397 23396 raft_consensus.cc:3055] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:23.115746 23396 raft_consensus.cc:740] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:23.116647 23396 consensus_queue.cc:260] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:23.121886 23396 raft_consensus.cc:3060] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:16:23.132979 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 122837us
W20260709 12:16:23.141680 31013 log.cc:927] Time spent T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.076s	user 0.003s	sys 0.000s
I20260709 12:16:23.151486 23391 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Committing config change with OpId 23.19: config changed from index 7 to 19, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.156715 23466 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.167016 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:23.171687 23396 raft_consensus.cc:2468] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:23.163031 31010 raft_consensus.cc:697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:23.145417 30961 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.172777 30958 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42236 (request call id 0) took 1349 ms. Trace:
W20260709 12:16:23.194067 31009 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.212s	user 0.000s	sys 0.003s
I20260709 12:16:23.183079 30958 rpcz_store.cc:276] 0709 12:16:21.817022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:21.817921 (+   899us) service_pool.cc:224] Handling call
0709 12:16:21.862349 (+ 44428us) log.cc:844] Serialized 508 byte log entry
0709 12:16:22.353074 (+490725us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b2c00028668. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a7f6da11 00007ff0a7f7ca0a 00007ff0a7f7c693 00007ff0a7f7c631 00007ff0a96b0b52 00007ff0a96b0cf5 00007ff0a95ed691 00007ff0a95cdeaa 00007ff0a95d4d53 00007ff0a95d4cd9 00007ff0a95d4c69
0709 12:16:23.163735 (+810661us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45964,"raft.queue_time_us":79862,"raft.run_cpu_time_us":6118,"raft.run_wall_time_us":335614,"spinlock_wait_cycles":11300608,"thread_start_us":163176,"threads_started":1,"wal-append.queue_time_us":246098}
I20260709 12:16:23.201050 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 87e270393fc841ddac0d9170b0bc6554 with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:23.206157 23214 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44670 (request call id 334) took 1752 ms. Trace:
I20260709 12:16:23.175248 30993 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:23.206591 23214 rpcz_store.cc:276] 0709 12:16:21.454249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:21.469799 (+ 15550us) service_pool.cc:224] Handling call
0709 12:16:21.488571 (+ 18772us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b5400002490. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a640fdb1 00007ff0a654fb71 00007ff0a65500b2 00007ff0af4c9759 00007ff0af461a76 00007ff0af432d58 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc
0709 12:16:23.206095 (+1717524us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:22.596703 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:22.597285 (+   582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:22.597323 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:16:22.597343 (+    20us) tablet.cc:662] Decoding operations
0709 12:16:22.605669 (+  8326us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:22.605736 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:22.605769 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:16:22.606178 (+   409us) tablet.cc:701] Row locks acquired
0709 12:16:22.606198 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:16:22.606348 (+   150us) write_op.cc:270] Start()
0709 12:16:22.606462 (+   114us) write_op.cc:276] Timestamp: P: 1783599382606298 usec, L: 0
0709 12:16:22.606487 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:16:22.607157 (+   670us) log.cc:844] Serialized 1375 byte log entry
0709 12:16:22.885240 (+278083us) op_driver.cc:464] REPLICATION: finished
0709 12:16:22.989873 (+104633us) write_op.cc:301] APPLY: starting
0709 12:16:22.989966 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:23.036611 (+ 46645us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:23.036731 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:23.038731 (+  2000us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:23.038945 (+   214us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 12:16:23.038992 (+    47us) write_op.cc:312] APPLY: finished
0709 12:16:23.039199 (+   207us) log.cc:844] Serialized 41 byte log entry
0709 12:16:23.039389 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:23.057312 (+ 17923us) write_op.cc:454] Released schema lock
0709 12:16:23.059986 (+  2674us) write_op.cc:341] FINISH: Updating metrics
0709 12:16:23.073850 (+ 13864us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:23.078298 (+  4448us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":702891,"reported_tablets":3,"spinlock_wait_cycles":13810432,"child_traces":[["op",{"apply.queue_time_us":104350,"apply.run_cpu_time_us":6890,"apply.run_wall_time_us":80629,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16679,"num_ops":3,"prepare.queue_time_us":274190,"prepare.run_cpu_time_us":5988,"prepare.run_wall_time_us":36153,"raft.queue_time_us":139196,"raft.run_cpu_time_us":12176,"raft.run_wall_time_us":108229,"replication_time_us":273623,"spinlock_wait_cycles":18304,"thread_start_us":365852,"threads_started":4,"wal-append.queue_time_us":9204}]]}
I20260709 12:16:23.238019 24010 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:16:23.238960 31010 consensus_queue.cc:237] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:23.247864 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:23.259883 23287 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:23.272576 30967 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:23.281404 24009 raft_consensus.cc:2955] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Committing config change with OpId 23.19: config changed from index 7 to 19, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.286405 23313 raft_consensus.cc:1275] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:16:23.289780 30905 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:23.303859 23392 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.316635 30905 consensus_queue.cc:1048] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:16:23.342314 23218 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 236) took 1970 ms. Trace:
I20260709 12:16:23.342900 23218 rpcz_store.cc:276] 0709 12:16:21.371581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:21.371797 (+   216us) service_pool.cc:224] Handling call
0709 12:16:21.400819 (+ 29022us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b5400002438. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af432e43 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:23.342214 (+1941395us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70992,"reported_tablets":3,"spinlock_wait_cycles":40029056}
I20260709 12:16:23.345790 23215 catalog_manager.cc:5697] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:23.381599 23462 raft_consensus.cc:1240] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 LEADER]: Rejecting Update request from peer 6253b834ffb744579b46fcede2fc0e91 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:23.412880 30894 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:23.413494 30894 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:23.422269 23313 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:23.432320 30983 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:23.434123 23215 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44678 (request call id 351) took 2134 ms. Trace:
I20260709 12:16:23.434428 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "2778b449761a46708f2f643b1b5b5854" is_pre_election: true
I20260709 12:16:23.438174 23215 rpcz_store.cc:276] 0709 12:16:21.299806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:21.312099 (+ 12293us) service_pool.cc:224] Handling call
0709 12:16:21.391496 (+ 79397us) spinlock_profiling.cc:243] Waited 53.3 ms on lock 0x7b5400002438. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0af44fab1 00007ff0af432e43 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6 00007ff0a79ae26f
0709 12:16:21.393124 (+  1628us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b5400001ed4. stack: 00007ff0a6555343 00007ff0a6555240 00007ff0a5ee41d4 00007ff0afc22304 00007ff0afc22299 00007ff0a95de090 00007ff0a967cf3a 00007ff0af432e9c 00007ff0af53104e 00007ff0ad9f22a4 00007ff0ad9f2232 00007ff0ad9f2161 00007ff0ad9f20dc 00007ff0ad9f1392 00007ff0a79af48c 00007ff0a79ae8c6
0709 12:16:23.413534 (+2020410us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:23.413564 (+    30us) write_op.cc:454] Released schema lock
0709 12:16:23.434064 (+ 20500us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:16:23.389771 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:16:23.390347 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:16:23.390376 (+    29us) write_op.cc:435] Acquired schema lock
0709 12:16:23.390392 (+    16us) tablet.cc:662] Decoding operations
0709 12:16:23.390761 (+   369us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:16:23.390834 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 12:16:23.390863 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:16:23.391049 (+   186us) tablet.cc:701] Row locks acquired
0709 12:16:23.391066 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:16:23.391209 (+   143us) write_op.cc:270] Start()
0709 12:16:23.391319 (+   110us) write_op.cc:276] Timestamp: P: 1783599383391158 usec, L: 0
0709 12:16:23.391350 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:16:23.392039 (+   689us) log.cc:844] Serialized 544 byte log entry
0709 12:16:23.394040 (+  2001us) op_driver.cc:464] REPLICATION: finished
0709 12:16:23.394498 (+   458us) write_op.cc:301] APPLY: starting
0709 12:16:23.394580 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:16:23.395436 (+   856us) tablet.cc:1370] finished BulkCheckPresence
0709 12:16:23.395540 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:16:23.396277 (+   737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:16:23.396464 (+   187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 12:16:23.396523 (+    59us) write_op.cc:312] APPLY: finished
0709 12:16:23.396718 (+   195us) log.cc:844] Serialized 25 byte log entry
0709 12:16:23.396898 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:16:23.397001 (+   103us) write_op.cc:454] Released schema lock
0709 12:16:23.397045 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":33450,"reported_tablets":2,"spinlock_wait_cycles":124975104,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":2835,"apply.run_wall_time_us":2834,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":36332,"prepare.run_cpu_time_us":2610,"prepare.run_wall_time_us":2609,"replication_time_us":2570,"spinlock_wait_cycles":41856,"thread_start_us":60067,"threads_started":1}]]}
I20260709 12:16:23.455152 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:23.457924 23436 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:23.464041 30879 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:23.464448 30894 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:23.486045 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 98182.9us
I20260709 12:16:23.495981 23396 raft_consensus.cc:1275] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:23.513024 24011 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 13 to 15, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.521574 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:23.545714 23396 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:16:23.553210 31010 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 12:16:23.592447 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:23.623163 23218 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.625089 23218 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 13 to 15, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:23.627979 23218 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 13 to 14, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } attrs { replace: true } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:23.650417 23320 raft_consensus.cc:1275] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:16:23.662436 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 105205us
I20260709 12:16:23.681630 30976 consensus_queue.cc:1048] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:23.738011 30930 consensus_queue.cc:1048] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:16:23.775312 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:23.776180 23392 raft_consensus.cc:606] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:23.838971 23218 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 7 to 19, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 23 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:23.859859 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:23.860587 23316 raft_consensus.cc:606] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Received request to transfer leadership to TS 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:23.866084 23392 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:16:23.868400 31010 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:23.870537 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b055a413d01c460895f3bc5034afc461"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:23.871222 23316 raft_consensus.cc:606] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:23.874764 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:23.875331 23316 raft_consensus.cc:606] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:23.898766 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:23.899655 23316 raft_consensus.cc:606] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:23.901600 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 144427us
I20260709 12:16:23.926412 23320 raft_consensus.cc:1275] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:16:23.955240 23320 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
mode: GRACEFUL
new_leader_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:42252
I20260709 12:16:23.955957 23320 raft_consensus.cc:606] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Received request to transfer leadership to TS da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:23.959626 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:16:23.984244 30993 raft_consensus.cc:993] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.022140 30993 raft_consensus.cc:1081] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
W20260709 12:16:24.057222 31040 log.cc:927] Time spent T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.080s	user 0.004s	sys 0.000s
I20260709 12:16:24.058799 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:16:24.060019 23213 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:36890 (request call id 223) took 31396 ms (client timeout 15000 ms). Trace:
W20260709 12:16:24.060622 23213 rpcz_store.cc:269] 0709 12:15:52.664094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:15:52.664443 (+   349us) service_pool.cc:224] Handling call
0709 12:16:24.059895 (+31395452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1786884,"reported_tablets":38,"spinlock_wait_cycles":9472}
I20260709 12:16:24.059962 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.069989 23320 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.075943 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 87e270393fc841ddac0d9170b0bc6554 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 19) from {username='slave'} at 127.0.0.1:43504
I20260709 12:16:24.090355 23214 catalog_manager.cc:5697] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 23 to 24, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 24 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:24.099093 23214 catalog_manager.cc:5697] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 10 to 11, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 11 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:24.114012 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 151696us
I20260709 12:16:24.143699 30905 consensus_queue.cc:1048] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:24.166594 23466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b055a413d01c460895f3bc5034afc461"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:16:24.167215 23466 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.167557 23466 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:24.183542 24010 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:16:24.185365 23215 catalog_manager.cc:5697] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 19 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 20 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:24.187562 23466 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.188585 23215 catalog_manager.cc:5697] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 20 to 21, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 21 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:24.192711 23392 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.195091 23469 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:16:24.203946 23466 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:24.209267 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:24.210002 24012 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:24.212926 30911 raft_consensus.cc:993] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.217546 24012 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:24.218727 23211 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44674 (request call id 326) took 1825 ms. Trace:
I20260709 12:16:24.221388 23211 rpcz_store.cc:276] 0709 12:16:22.384974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:16:22.385133 (+   159us) service_pool.cc:224] Handling call
0709 12:16:24.210360 (+1825227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23618,"reported_tablets":7}
I20260709 12:16:24.205129 31045 tablet_replica.cc:333] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:24.227769 30961 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:16:24.226162 30911 raft_consensus.cc:1081] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.227106 23436 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:24.237406 30930 raft_consensus.cc:2804] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:24.243873 23469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36038
I20260709 12:16:24.244560 23469 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.244894 23469 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:24.254696 30930 raft_consensus.cc:697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:24.255851 30930 consensus_queue.cc:237] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.256400 31039 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:24.260305 31045 raft_consensus.cc:2243] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:24.286736 31045 raft_consensus.cc:2272] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:24.291198 31045 ts_tablet_manager.cc:1916] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:24.295770 30993 raft_consensus.cc:993] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.296375 30993 raft_consensus.cc:1081] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.284909 23469 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.328791 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:24.341411 23462 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:24.345340 23392 raft_consensus.cc:3055] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:16:24.345750 23392 raft_consensus.cc:740] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:24.346473 23392 consensus_queue.cc:260] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.347287 23392 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:24.351110 24012 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:24.353596 30994 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:16:24.356949 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 104029us
I20260709 12:16:24.357205 23469 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:24.360211 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d83783f67774af1b007547d0aca8da1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:24.360849 24010 raft_consensus.cc:3060] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:24.379070 23392 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 25.
I20260709 12:16:24.385800 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:24.386471 23312 raft_consensus.cc:3055] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:16:24.386814 23312 raft_consensus.cc:740] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:24.387574 23312 consensus_queue.cc:260] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.388681 23312 raft_consensus.cc:3060] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:24.368266 30986 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:16:24.399034 24010 raft_consensus.cc:2468] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 25.
I20260709 12:16:24.405388 23436 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:24.411895 30879 raft_consensus.cc:2804] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:24.412456 30879 raft_consensus.cc:697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 25 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:24.413373 30879 consensus_queue.cc:237] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.419288 23312 raft_consensus.cc:2468] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:24.444517 23214 catalog_manager.cc:5697] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 15, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.488444 23463 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:16:24.489050 23463 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.489459 23463 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:24.499109 23463 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.513743 23462 raft_consensus.cc:1240] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 14. Current term is 15. Ops: []
I20260709 12:16:24.521878 23463 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:24.523368 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:24.524271 24010 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:24.532403 23312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:24.533491 23312 raft_consensus.cc:3055] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:16:24.535161 23312 raft_consensus.cc:740] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:24.535903 23312 consensus_queue.cc:260] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.536991 23312 raft_consensus.cc:3060] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:24.547178 23312 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:16:24.547896 23215 catalog_manager.cc:5697] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 24 to 25, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 25 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.552642 24010 raft_consensus.cc:2468] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:16:24.558080 23436 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:24.561884 30906 raft_consensus.cc:993] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.560804 30993 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:24.562413 30906 raft_consensus.cc:1081] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.556862 23391 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:24.581825 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44055.1us
I20260709 12:16:24.584587 30905 raft_consensus.cc:2804] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:24.615513 31043 raft_consensus.cc:993] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:16:24.621726 31042 raft_consensus.cc:1081] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Signalling peer 54640ab8abeb4c949426d66c7e5aed87 to start an election
I20260709 12:16:24.625317 30958 raft_consensus.cc:993] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: : Instructing follower da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.625913 30958 raft_consensus.cc:1081] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Signalling peer da1e2fd157894634bf92d6c82c06ec64 to start an election
I20260709 12:16:24.644852 23204 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cfaa0d3f687341a1ac860baa49e8bfb4 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:24.617873 30905 raft_consensus.cc:697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:24.661268 30905 consensus_queue.cc:237] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.671808 24012 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:24.674872 31051 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:24.671749 23469 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:24.685909 23463 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:24.698155 24012 raft_consensus.cc:2955] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:24.701030 23218 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index 15 to 16, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 19 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:24.727694 30986 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:24.739562 30937 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:16:24.759400 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fa5f71c212664b78a4deb373bf5ed2f5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:24.766067 24010 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "709006b08c044ed38e2df81795ed1613"
dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
 from {username='slave'} at 127.0.0.1:53940
I20260709 12:16:24.766680 24010 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.767052 24010 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:24.765028 23215 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 12 to 13, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:24.826022 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet cfaa0d3f687341a1ac860baa49e8bfb4 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:33622
I20260709 12:16:24.828622 24009 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:24.830510 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50747us
I20260709 12:16:24.834527 24010 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.839047 24015 raft_consensus.cc:1240] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 20. Current term is 21. Ops: []
I20260709 12:16:24.848347 24010 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:24.848994 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:24.849706 23316 raft_consensus.cc:3055] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:16:24.849996 23316 raft_consensus.cc:740] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:24.850687 23316 consensus_queue.cc:260] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:24.856925 23316 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:24.883139 23316 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:24.884845 23980 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:24.886430 31036 raft_consensus.cc:2804] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:24.886960 31036 raft_consensus.cc:697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:24.887773 31036 consensus_queue.cc:237] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:24.889797 31061 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:16:24.892825 31045 ts_tablet_manager.cc:1929] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.18
I20260709 12:16:24.870436 23214 catalog_manager.cc:5697] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 14 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:24.908594 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 169449us
I20260709 12:16:24.908938 31071 tablet_replica.cc:333] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:24.915076 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:25.013499 23391 raft_consensus.cc:3060] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:24.901039 31045 log.cc:1199] T 87e270393fc841ddac0d9170b0bc6554 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/87e270393fc841ddac0d9170b0bc6554
I20260709 12:16:25.019397 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 87e270393fc841ddac0d9170b0bc6554 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:16:24.976816 31067 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:16:24.998212 31003 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.085s	user 0.003s	sys 0.000s
W20260709 12:16:24.999893 31066 log.cc:927] Time spent T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
W20260709 12:16:25.030472 31009 log.cc:927] Time spent T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.114s	user 0.002s	sys 0.001s
I20260709 12:16:25.029608 31051 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:25.031577 24012 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:25.038137 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet fa5f71c212664b78a4deb373bf5ed2f5 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:42236
I20260709 12:16:25.048530 23391 raft_consensus.cc:2468] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:25.055802 23469 raft_consensus.cc:2955] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:25.066021 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:16:25.066627 23462 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.066946 23462 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:25.074492 23462 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:25.076541 23462 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:25.081856 23463 raft_consensus.cc:1240] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Rejecting Update request from peer 2778b449761a46708f2f643b1b5b5854 for earlier term 23. Current term is 24. Ops: []
I20260709 12:16:25.083580 23469 raft_consensus.cc:2955] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Committing config change with OpId 19.16: config changed from index 15 to 16, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:25.088558 23462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
 from {username='slave'} at 127.0.0.1:36042
I20260709 12:16:25.089203 23462 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.089643 23462 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:25.093602 23391 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:16:25.095739 31071 raft_consensus.cc:2243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:25.096621 31071 raft_consensus.cc:2272] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:25.099831 23316 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:25.102030 31071 ts_tablet_manager.cc:1916] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:25.103825 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 122111us
I20260709 12:16:25.106132 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:25.106717 23316 raft_consensus.cc:3055] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:25.106988 23316 raft_consensus.cc:740] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:25.107841 23316 consensus_queue.cc:260] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:25.108927 23316 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:25.120059 23316 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 24.
I20260709 12:16:25.131201 23462 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:25.141907 31069 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:25.145272 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2984a5ac8b6b49c9aa8eb969a484d610 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:25.148972 23434 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:25.150549 31069 raft_consensus.cc:2804] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:25.151713 31069 raft_consensus.cc:697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:25.153102 31069 consensus_queue.cc:237] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:25.154467 31076 tablet_replica.cc:333] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:25.159273 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:25.160050 23396 raft_consensus.cc:3060] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:25.170506 23462 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:25.156628 23320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:25.171629 23320 raft_consensus.cc:3055] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:16:25.172021 23320 raft_consensus.cc:740] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:25.172909 23320 consensus_queue.cc:260] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:25.174464 23320 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:25.175443 31076 raft_consensus.cc:2243] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:25.164836 23214 catalog_manager.cc:5697] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:25.193905 24010 raft_consensus.cc:1275] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:25.201318 23215 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 14 to 15, VOTER 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) evicted. New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:25.205468 23313 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:25.207736 31068 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:25.226192 31071 ts_tablet_manager.cc:1929] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.15
I20260709 12:16:25.226682 31071 log.cc:1199] T cfaa0d3f687341a1ac860baa49e8bfb4 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/cfaa0d3f687341a1ac860baa49e8bfb4
I20260709 12:16:25.228255 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet cfaa0d3f687341a1ac860baa49e8bfb4 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:25.233584 23320 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:16:25.236187 23396 raft_consensus.cc:2468] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 24.
I20260709 12:16:25.235996 23434 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:25.240036 24011 raft_consensus.cc:1275] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:16:25.201378 31076 raft_consensus.cc:2272] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:25.243366 31068 raft_consensus.cc:2804] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:16:25.246793 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:25.247365 24011 raft_consensus.cc:3060] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:16:25.261767 23309 tablet_service.cc:1558] Processing DeleteTablet for tablet 2984a5ac8b6b49c9aa8eb969a484d610 with delete_type TABLET_DATA_TOMBSTONED (TS 2778b449761a46708f2f643b1b5b5854 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:42236
I20260709 12:16:25.265497 31022 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:16:25.249006 31068 raft_consensus.cc:697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:25.260030 31076 ts_tablet_manager.cc:1916] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:25.276408 31068 consensus_queue.cc:237] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:25.289628 24011 raft_consensus.cc:2468] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
I20260709 12:16:25.309383 23211 catalog_manager.cc:5697] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 23 to 24, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 24 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:25.318256 31076 ts_tablet_manager.cc:1929] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.12
I20260709 12:16:25.325843 31076 log.cc:1199] T fa5f71c212664b78a4deb373bf5ed2f5 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/fa5f71c212664b78a4deb373bf5ed2f5
I20260709 12:16:25.328619 31076 tablet_replica.cc:333] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:25.328550 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet fa5f71c212664b78a4deb373bf5ed2f5 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:25.330214 31076 raft_consensus.cc:2243] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:25.333904 31076 raft_consensus.cc:2272] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:25.346606 31076 ts_tablet_manager.cc:1916] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:25.360000 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 74105.7us
I20260709 12:16:25.376058 23316 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:16:25.427870 24011 raft_consensus.cc:1275] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:25.383599 30976 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:16:25.437126 23211 catalog_manager.cc:5697] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 13 to 14, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 14 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:25.445798 31078 log.cc:927] Time spent T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:16:25.455641 31022 consensus_queue.cc:1048] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:16:25.459694 30879 consensus_queue.cc:1048] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:16:25.490681 31082 consensus_queue.cc:1048] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:16:25.500182 23392 raft_consensus.cc:1275] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:16:25.530109 31083 log.cc:927] Time spent T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:16:25.510452 30983 consensus_queue.cc:1048] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:16:25.536149 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.682973 23462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:16:25.684062 23462 raft_consensus.cc:606] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:25.739619 31076 ts_tablet_manager.cc:1929] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.14
I20260709 12:16:25.786616 31076 log.cc:1199] T 2984a5ac8b6b49c9aa8eb969a484d610 P 2778b449761a46708f2f643b1b5b5854: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/wals/2984a5ac8b6b49c9aa8eb969a484d610
I20260709 12:16:25.801108 23463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:16:25.802153 23463 raft_consensus.cc:606] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:25.807791 23204 catalog_manager.cc:5028] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): tablet 2984a5ac8b6b49c9aa8eb969a484d610 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:25.843195 23463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:16:25.844270 23463 raft_consensus.cc:606] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:25.847163 24012 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:25.850234 23396 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:25.852687 23320 raft_consensus.cc:1275] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:16:25.859200 31022 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:25.861963 23463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0"
dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
mode: GRACEFUL
new_leader_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:36140
I20260709 12:16:25.862624 23463 raft_consensus.cc:606] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Received request to transfer leadership to TS 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:25.865607 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:25.861984 31097 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:16:25.897702 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:25.898869 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:25.927189 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 65694.5us
I20260709 12:16:25.945547 31097 consensus_queue.cc:1048] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:25.951457 31017 raft_consensus.cc:993] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:25.951970 31017 raft_consensus.cc:1081] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:25.970250 23312 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42792
I20260709 12:16:25.970875 23312 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.971228 23312 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:25.979991 23312 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.101853 23316 raft_consensus.cc:1275] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:26.102845 23312 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:26.124753 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:26.126186 23463 raft_consensus.cc:3055] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:26.126586 23463 raft_consensus.cc:740] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:26.133023 30930 consensus_queue.cc:1048] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:26.128434 23463 consensus_queue.cc:260] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.124739 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:26.154088 24010 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:26.155318 23463 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:16:26.231487 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 56894.3us
I20260709 12:16:26.239696 31096 raft_consensus.cc:993] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.240566 30930 raft_consensus.cc:1081] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.259690 24010 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:26.262825 23287 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:26.265352 30961 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:26.265959 30961 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:26.271341 31017 raft_consensus.cc:993] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.271449 31096 raft_consensus.cc:993] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: : Instructing follower 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.272541 31068 raft_consensus.cc:1081] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.273113 31022 raft_consensus.cc:1081] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Signalling peer 6253b834ffb744579b46fcede2fc0e91 to start an election
I20260709 12:16:26.277607 23388 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:16:26.278379 23388 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.278782 23388 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:26.295151 23392 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:16:26.296010 23392 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.296518 23392 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:26.297101 23391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
 from {username='slave'} at 127.0.0.1:52270
I20260709 12:16:26.297757 23391 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.298127 23391 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:26.267138 30961 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.312386 23388 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:26.344996 23388 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:26.347831 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:26.348562 23461 raft_consensus.cc:3055] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:16:26.348956 23461 raft_consensus.cc:740] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:26.353648 23461 consensus_queue.cc:260] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:26.355019 23461 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:26.380779 23461 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:26.383085 23362 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:26.387949 23463 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 17.
I20260709 12:16:26.397794 23391 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.399277 23392 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.399699 31089 raft_consensus.cc:2804] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:16:26.404184 31089 raft_consensus.cc:697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:26.405248 31089 consensus_queue.cc:237] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:26.424430 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:26.425235 23466 raft_consensus.cc:3055] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:16:26.425671 23466 raft_consensus.cc:740] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:26.430583 23392 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:26.426705 23466 consensus_queue.cc:260] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.430886 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:26.431944 23391 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Requested vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:26.432148 24011 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:26.427877 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:26.432957 23461 raft_consensus.cc:3055] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:16:26.433688 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:26.463610 24010 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:26.462776 23466 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:26.433279 23461 raft_consensus.cc:740] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:26.433612 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "2778b449761a46708f2f643b1b5b5854"
I20260709 12:16:26.441144 23211 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 16 to 17, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.479547 24011 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:26.455667 23215 catalog_manager.cc:5697] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 16 to 17, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 17 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.488415 24010 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
I20260709 12:16:26.491802 23362 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:26.495546 23316 raft_consensus.cc:3060] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:26.498242 23461 consensus_queue.cc:260] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.499897 23461 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:26.501140 23362 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:26.502235 31051 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:26.502759 31051 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:26.503803 31051 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.505745 31052 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:26.506351 31052 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:26.507429 31052 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:26.578471 23466 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:26.588267 23316 raft_consensus.cc:2468] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 17.
I20260709 12:16:26.612234 23461 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
I20260709 12:16:26.717118 23461 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:26.730142 30961 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:26.739476 24010 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:16:26.762367 31042 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:16:26.844002 23392 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:26.880679 23466 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:26.882728 31051 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:26.907517 23215 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 23 to 24, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 24 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.909871 23215 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 17 to 18, leader changed from da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 18 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.921936 23466 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:26.924733 30994 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:16:26.926715 23362 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97dc7e2c632a45499346d6fcf8e4aee9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:26.927551 24015 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:16:26.931198 31051 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:26.947432 23314 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:16:26.949970 31051 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:26.972963 23313 raft_consensus.cc:1275] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:26.980007 30994 consensus_queue.cc:1048] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:26.983232 23462 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:16:26.986122 30994 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:27.033658 24011 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:16:27.041579 30938 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:16:27.044687 23469 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:16:27.052475 31051 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:27.102620 30994 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.119989 23314 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.123358 23462 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.192718 23215 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 reported cstate change: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.259919 23316 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:27.284864 23466 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:16:27.290235 23396 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:16:27.294430 31015 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:16:27.298170 23287 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b87a4fcda7f4296b7f91f6ba3a14f29. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:27.299326 31042 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:16:27.360773 31042 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.383502 23396 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
W20260709 12:16:27.439337 31133 log.cc:927] Time spent T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Append to log took a long time: real 0.055s	user 0.000s	sys 0.006s
I20260709 12:16:27.459991 23466 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.479252 31138 ts_tablet_manager.cc:933] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:27.548018 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 69602us
I20260709 12:16:27.555555 23215 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: config changed from index -1 to 11, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.579890 23316 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:27.592584 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.646117 31138 tablet_copy_client.cc:323] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.194:41721
I20260709 12:16:27.653918 23406 tablet_copy_service.cc:140] P 6253b834ffb744579b46fcede2fc0e91: Received BeginTabletCopySession request for tablet 97dc7e2c632a45499346d6fcf8e4aee9 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:34554)
I20260709 12:16:27.654598 23406 tablet_copy_service.cc:161] P 6253b834ffb744579b46fcede2fc0e91: Beginning new tablet copy session on tablet 97dc7e2c632a45499346d6fcf8e4aee9 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:34554: session id = 54640ab8abeb4c949426d66c7e5aed87-97dc7e2c632a45499346d6fcf8e4aee9
I20260709 12:16:27.675024 23406 tablet_copy_source_session.cc:215] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:27.680980 23392 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:27.681821 23469 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:16:27.683017 31015 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:27.684834 31015 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:27.693229 23287 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 -> Peer 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405): Couldn't send request to peer 54640ab8abeb4c949426d66c7e5aed87. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 71b8a974ecc34bd487fe9c208e6acc17. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:27.707973 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 48778.6us
I20260709 12:16:27.730201 31137 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.734477 23396 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.741660 23469 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index -1 to 12, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.751678 23392 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
mode: GRACEFUL
new_leader_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:45800
I20260709 12:16:27.752614 23392 raft_consensus.cc:606] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Received request to transfer leadership to TS 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:27.752113 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index -1 to 12, NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) added. New cstate: current_term: 15 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } } }
I20260709 12:16:27.758116 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:16:27.763324 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:27.772439 31138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dc7e2c632a45499346d6fcf8e4aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:27.828894 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.839619 31052 raft_consensus.cc:993] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: : Instructing follower 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:27.840153 30938 raft_consensus.cc:1081] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Signalling peer 2778b449761a46708f2f643b1b5b5854 to start an election
I20260709 12:16:27.842265 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "baab486079784de4933d5f96a0a5321b"
dest_uuid: "2778b449761a46708f2f643b1b5b5854"
 from {username='slave'} at 127.0.0.1:42788
I20260709 12:16:27.842813 23316 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:27.843182 23316 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:27.845111 31138 tablet_copy_client.cc:806] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:27.867347 31138 tablet_copy_client.cc:670] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:27.849980 23316 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:27.876010 23316 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:27.878286 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:27.878986 23391 raft_consensus.cc:3055] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:16:27.879312 23391 raft_consensus.cc:740] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:27.880272 23391 consensus_queue.cc:260] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:27.881812 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: true dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:27.882427 24010 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:27.908181 23391 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:27.934901 31138 tablet_copy_client.cc:538] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:27.924214 24010 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:27.959836 23287 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87; no voters: 
I20260709 12:16:27.962445 31147 ts_tablet_manager.cc:933] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:27.963282 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2778b449761a46708f2f643b1b5b5854 in term 20.
I20260709 12:16:27.971915 31138 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:27.972695 31146 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:27.977663 31146 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Becoming Leader. State: Replica: 2778b449761a46708f2f643b1b5b5854, State: Running, Role: LEADER
I20260709 12:16:27.979174 31147 tablet_copy_client.cc:323] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:16:27.978853 31146 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:27.982914 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:16:27.983614 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-8b87a4fcda7f4296b7f91f6ba3a14f29
I20260709 12:16:28.005065 23331 tablet_copy_source_session.cc:215] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:28.009131 31147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:28.026149 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 53045.1us
I20260709 12:16:28.081830 23215 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 reported cstate change: term changed from 19 to 20, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to 2778b449761a46708f2f643b1b5b5854 (127.21.150.193). New cstate: current_term: 20 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:28.084079 31138 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:28.085114 31138 tablet_bootstrap.cc:492] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:28.087387 31138 ts_tablet_manager.cc:1403] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.022s
I20260709 12:16:28.146507 31151 ts_tablet_manager.cc:933] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Initiating tablet copy from peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055)
I20260709 12:16:28.155862 31138 raft_consensus.cc:359] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.156823 31138 raft_consensus.cc:740] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:28.157542 31138 consensus_queue.cc:260] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.160777 31138 ts_tablet_manager.cc:1434] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.073s	user 0.008s	sys 0.000s
I20260709 12:16:28.198758 23406 tablet_copy_service.cc:342] P 6253b834ffb744579b46fcede2fc0e91: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-97dc7e2c632a45499346d6fcf8e4aee9 received from {username='slave'} at 127.0.0.1:34554
I20260709 12:16:28.199255 23406 tablet_copy_service.cc:434] P 6253b834ffb744579b46fcede2fc0e91: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-97dc7e2c632a45499346d6fcf8e4aee9 on tablet 97dc7e2c632a45499346d6fcf8e4aee9 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:28.241911 31151 tablet_copy_client.cc:323] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Beginning tablet copy session from remote peer at address 127.21.150.193:46055
I20260709 12:16:28.243849 23331 tablet_copy_service.cc:140] P 2778b449761a46708f2f643b1b5b5854: Received BeginTabletCopySession request for tablet 71b8a974ecc34bd487fe9c208e6acc17 from peer 54640ab8abeb4c949426d66c7e5aed87 ({username='slave'} at 127.0.0.1:43746)
I20260709 12:16:28.244323 23331 tablet_copy_service.cc:161] P 2778b449761a46708f2f643b1b5b5854: Beginning new tablet copy session on tablet 71b8a974ecc34bd487fe9c208e6acc17 from peer 54640ab8abeb4c949426d66c7e5aed87 at {username='slave'} at 127.0.0.1:43746: session id = 54640ab8abeb4c949426d66c7e5aed87-71b8a974ecc34bd487fe9c208e6acc17
I20260709 12:16:28.256014 23331 tablet_copy_source_session.cc:215] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:16:28.260439 31151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71b8a974ecc34bd487fe9c208e6acc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:28.292704 31151 tablet_copy_client.cc:806] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:28.293601 31151 tablet_copy_client.cc:670] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:28.306849 31151 tablet_copy_client.cc:538] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:28.294907 31147 tablet_copy_client.cc:806] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 0 data blocks...
I20260709 12:16:28.311730 31147 tablet_copy_client.cc:670] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Starting download of 1 WAL segments...
I20260709 12:16:28.321790 31147 tablet_copy_client.cc:538] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:16:28.325161 31151 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:28.339841 24012 raft_consensus.cc:1217] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 12:16:28.398552 31147 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap starting.
I20260709 12:16:28.475627 31151 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:28.476630 31151 tablet_bootstrap.cc:492] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:28.477324 31151 ts_tablet_manager.cc:1403] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.152s	user 0.041s	sys 0.005s
I20260709 12:16:28.480409 31151 raft_consensus.cc:359] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.481230 31151 raft_consensus.cc:740] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:28.491750 31151 consensus_queue.cc:260] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.495193 23392 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:16:28.498385 30919 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:28.502761 31151 ts_tablet_manager.cc:1434] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:16:28.504768 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-71b8a974ecc34bd487fe9c208e6acc17 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:28.505254 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-71b8a974ecc34bd487fe9c208e6acc17 on tablet 71b8a974ecc34bd487fe9c208e6acc17 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:28.511756 31147 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:28.512853 31147 tablet_bootstrap.cc:492] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Bootstrap complete.
I20260709 12:16:28.513619 31147 ts_tablet_manager.cc:1403] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Time spent bootstrapping tablet: real 0.115s	user 0.034s	sys 0.012s
I20260709 12:16:28.516611 31147 raft_consensus.cc:359] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.517468 31147 raft_consensus.cc:740] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Initialized, Role: LEARNER
I20260709 12:16:28.518168 24010 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:16:28.518105 31147 consensus_queue.cc:260] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: NON_VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: true } }
I20260709 12:16:28.521813 30919 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:16:28.522486 31147 ts_tablet_manager.cc:1434] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:28.533982 23331 tablet_copy_service.cc:342] P 2778b449761a46708f2f643b1b5b5854: Request end of tablet copy session 54640ab8abeb4c949426d66c7e5aed87-8b87a4fcda7f4296b7f91f6ba3a14f29 received from {username='slave'} at 127.0.0.1:43746
I20260709 12:16:28.534772 23331 tablet_copy_service.cc:434] P 2778b449761a46708f2f643b1b5b5854: ending tablet copy session 54640ab8abeb4c949426d66c7e5aed87-8b87a4fcda7f4296b7f91f6ba3a14f29 on tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 with peer 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:28.706113 24010 raft_consensus.cc:1217] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.11->[15.12-15.12]   Dedup: 15.12->[]
I20260709 12:16:28.875519 24010 raft_consensus.cc:1217] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 12:16:28.916349 31052 raft_consensus.cc:1064] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:16:28.919091 31052 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:28.970130 24010 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 LEARNER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:28.974542 30938 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:28.982564 23462 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:28.982995 23316 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:16:28.984717 30938 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:16:28.985903 31089 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:29.031272 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 47251us
I20260709 12:16:29.036317 31165 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.061093 24012 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.062314 23316 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.075942 23215 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:29.087594 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.111515 23462 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.118359 30919 raft_consensus.cc:1064] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
I20260709 12:16:29.121119 30919 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:29.156805 24010 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:16:29.158589 23469 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:16:29.159013 23391 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:16:29.160979 31137 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:29.162922 31137 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:29.164870 31015 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:29.239411 23392 consensus_queue.cc:237] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:29.250495 24011 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:29.252135 31164 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:29.252889 23314 raft_consensus.cc:1275] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:16:29.259645 31165 consensus_queue.cc:1048] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:29.273728 30938 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.278556 24015 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.290946 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:29.292773 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:29.294083 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.322588 23316 raft_consensus.cc:2955] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.341806 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 97dc7e2c632a45499346d6fcf8e4aee9 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:16:29.370358 30919 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.420490 23211 catalog_manager.cc:5697] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: config changed from index 12 to 13, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 14 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:29.438139 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.527361 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 97dc7e2c632a45499346d6fcf8e4aee9 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:43504
W20260709 12:16:29.535796 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 164420us
I20260709 12:16:29.549211 23392 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.554066 24010 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.607438 31015 raft_consensus.cc:1064] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: attempting to promote NON_VOTER 54640ab8abeb4c949426d66c7e5aed87 to VOTER
W20260709 12:16:29.649236 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 89301.8us
W20260709 12:16:29.663399 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 98308.8us
I20260709 12:16:29.699450 23218 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 12 to 13, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:29.700695 31178 tablet_replica.cc:333] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:29.701792 31178 raft_consensus.cc:2243] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:29.702467 31178 raft_consensus.cc:2272] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:29.715435 31178 ts_tablet_manager.cc:1916] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:29.719087 31015 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:29.795023 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 111091us
I20260709 12:16:29.825297 23469 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
W20260709 12:16:29.820475 31169 log.cc:927] Time spent T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: Append to log took a long time: real 0.076s	user 0.002s	sys 0.005s
I20260709 12:16:29.838270 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
W20260709 12:16:29.914211 23204 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:29.919595 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 91943.5us
W20260709 12:16:30.028735 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 57788.9us
W20260709 12:16:30.034596 23204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 120579us
I20260709 12:16:30.023669 31178 ts_tablet_manager.cc:1929] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.12
I20260709 12:16:30.043185 31178 log.cc:1199] T 97dc7e2c632a45499346d6fcf8e4aee9 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/97dc7e2c632a45499346d6fcf8e4aee9
W20260709 12:16:30.050137 23204 catalog_manager.cc:4755] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): ChangeConfig:REMOVE_PEER RPC failed for tablet 71b8a974ecc34bd487fe9c208e6acc17: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:30.105652 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 70814.5us
I20260709 12:16:30.070112 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 97dc7e2c632a45499346d6fcf8e4aee9 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:16:30.116068 23206 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 45996.9us
W20260709 12:16:30.135354 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.136443 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.145248 23287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 297364us
W20260709 12:16:30.137704 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.159757 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.173372 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 90256.2us
W20260709 12:16:30.163921 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.190199 24010 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEARNER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:16:30.195189 23360 consensus_peers.cc:597] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.198081 23360 consensus_peers.cc:597] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.244602 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 46569.1us
W20260709 12:16:30.196131 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.247465 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 56456.2us
I20260709 12:16:30.258849 23392 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:16:30.246445 23360 consensus_peers.cc:597] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.265837 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.283957 23360 consensus_peers.cc:597] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.293972 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 133172us
W20260709 12:16:30.286804 23360 consensus_peers.cc:597] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.280860 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.302547 23360 consensus_peers.cc:597] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.303246 23360 consensus_peers.cc:597] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.304029 23360 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.342084 23360 consensus_peers.cc:597] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.347662 23462 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Refusing update from remote peer 2778b449761a46708f2f643b1b5b5854: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:16:30.349133 30919 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:16:30.349982 23360 consensus_peers.cc:597] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.350780 23360 consensus_peers.cc:597] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.376487 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.387568 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 89937.5us
W20260709 12:16:30.405524 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.406186 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.441571 31177 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:16:30.461943 30919 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:16:30.468668 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.473829 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.474472 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.474979 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.475689 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.476455 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.477130 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.483687 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.518574 31184 raft_consensus.cc:493] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:30.519201 31184 raft_consensus.cc:515] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.522609 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.532107 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e2145c1046c436e8ca4a54ef0df4ab5" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.521270 31184 leader_election.cc:290] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:30.550799 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 109999us
I20260709 12:16:30.548875 23287 leader_election.cc:304] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.512625 31137 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.560542 31184 raft_consensus.cc:2749] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:30.583087 31184 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:30.595608 24015 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:30.571664 31137 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.609513 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.615623 31015 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.616145 31015 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.618506 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a59619d146f04895b0e204f6e4bdcf47" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.629112 31015 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:30.631189 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.618002 30919 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:30.634701 30919 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.636952 30919 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.637326 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f55bd3b311de43ba8aca6be22937f68a" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.642606 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.643417 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d736b5901fbd4de09b83cc09a8d00fd1" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.615314 31137 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:30.647032 23392 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:30.693845 23287 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.702032 31137 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:30.703449 31137 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.704012 31137 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.704747 23287 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.705580 31177 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:30.706012 23287 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.706137 31137 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:30.706760 31177 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:30.708953 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.718283 23461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88fa82a6fcc0481fa12dd9a6f1782ab3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.730334 23287 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.734185 31177 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:30.751327 30919 raft_consensus.cc:493] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:30.751943 30919 raft_consensus.cc:515] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.755961 23461 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:30.760412 30919 leader_election.cc:290] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.762027 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.764180 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22fc2096e78f47158bbb4bae3a474e2c" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.773900 31137 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:30.774551 31137 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.778106 31137 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.780805 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.788900 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1399c0b792a048cda2bcae40b23432c0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.724555 23215 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 11 to 12, 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "2778b449761a46708f2f643b1b5b5854" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:30.792658 23287 leader_election.cc:304] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.800246 23287 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.794569 30919 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:30.805181 31183 raft_consensus.cc:493] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:30.805724 31183 raft_consensus.cc:515] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.807755 31183 leader_election.cc:290] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.801370 31189 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.808923 31189 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.803288 31201 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.810335 31201 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.811100 31189 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.804100 30919 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.800849 31177 raft_consensus.cc:2749] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:30.814229 31177 raft_consensus.cc:493] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:30.814245 30919 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.814733 31177 raft_consensus.cc:515] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.815361 30919 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:30.824640 30919 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.826781 30919 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.819494 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.812459 31201 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:30.821219 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.821498 31177 leader_election.cc:290] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.821851 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.822477 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6161f3a92cf44138f9a68a6f005c4e9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.830269 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.823966 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cd3243d8c2d447ca873f0890f2c0058" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.816820 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca56a943ba904b779d7584f94c266344" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.832051 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea456647290e4959b61a6fa23ecfbfd4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.834439 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.833976 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03644bf5f16347c98fd9217f8a7fab79" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.836730 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.837744 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.804189 31137 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:30.841949 23287 leader_election.cc:304] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.846546 31177 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.870841 31177 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.876432 23287 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.875970 31184 raft_consensus.cc:2749] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:30.873287 31177 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.883594 23287 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.877702 31184 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:30.896023 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.898828 23287 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.901703 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dc7e2c632a45499346d6fcf8e4aee9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.907771 23287 leader_election.cc:304] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:30.909759 31189 raft_consensus.cc:2749] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:30.913040 31184 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:30.913880 31177 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:30.927335 23287 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.930308 31184 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:30.933996 23287 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.936299 31184 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:30.948391 31184 raft_consensus.cc:493] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:30.949290 31184 raft_consensus.cc:515] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:30.952049 31184 leader_election.cc:290] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:30.958717 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:30.958770 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709006b08c044ed38e2df81795ed1613" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:30.966284 31177 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:30.974189 23287 leader_election.cc:304] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:30.975494 31201 raft_consensus.cc:2749] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:30.975467 31177 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:30.977794 31177 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:30.986498 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:30.986450 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d137914eb14302bb888a191819fb75" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.004129 23287 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.014307 31137 raft_consensus.cc:493] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:31.015085 31137 raft_consensus.cc:515] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.017663 31201 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:31.022181 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.024907 31137 leader_election.cc:290] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:31.034338 31137 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:31.034946 31137 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.037199 31137 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:31.040591 31137 raft_consensus.cc:493] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.041319 31184 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.041783 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.041960 31184 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.042351 31177 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:31.042868 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.042922 31177 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.041126 31137 raft_consensus.cc:515] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.044132 31184 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:31.042142 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e3b5eed804f2a8c2a01e5d1ae5f53" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.045019 31177 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.045646 31137 leader_election.cc:290] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.046883 31137 raft_consensus.cc:493] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.047387 31137 raft_consensus.cc:515] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.047783 31189 raft_consensus.cc:493] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.048314 31189 raft_consensus.cc:515] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.049700 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.049870 23287 leader_election.cc:304] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.050357 31189 leader_election.cc:290] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.050736 30919 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.051018 23287 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:31.051075 31189 raft_consensus.cc:2749] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:31.051266 30919 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.052325 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.053280 30919 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:31.055841 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.052351 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b055a413d01c460895f3bc5034afc461" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.058336 31137 leader_election.cc:290] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.059003 31201 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:31.061480 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.062649 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1376b20c4a2b47efbb5cd4afdb7f2ea4" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.062870 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5d8af85525845a192206b43a61f5c41" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.063211 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bbbae982b14adea2da3b5b5217b18e" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.062856 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.078962 31177 raft_consensus.cc:493] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:31.079608 31177 raft_consensus.cc:515] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.082378 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.083197 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9689b2f60f934eb6b1c067507423b816" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.083132 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba09297a7e5c4b4f8b873c36c70e2a2d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.085028 31177 leader_election.cc:290] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.092706 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.093151 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3dffd4f7c72413b84a3862372dcce85" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.114596 31189 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:31.115254 31189 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.117764 31189 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.118633 23287 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.119442 31177 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:31.120803 23287 leader_election.cc:304] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.124919 31177 raft_consensus.cc:2749] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:31.127085 23287 leader_election.cc:304] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.131695 31177 raft_consensus.cc:2749] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:31.132614 31177 raft_consensus.cc:493] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:31.133322 31177 raft_consensus.cc:515] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.135360 23287 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.136561 23287 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.136703 31189 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:31.137517 31189 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:16:31.144572 31177 leader_election.cc:290] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:31.148501 23204 catalog_manager.cc:4755] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): ChangeConfig:REMOVE_PEER RPC failed for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:31.153491 23287 leader_election.cc:304] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.154455 31177 raft_consensus.cc:2749] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:31.158587 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.162257 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.163818 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bc5916fff44ea3b1dcd31091d822b9" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.166567 23287 leader_election.cc:304] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.169731 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcbe5680eae64fa2a2565e9b0973703d" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.179862 23287 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.180685 30919 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:31.183699 23287 leader_election.cc:304] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.184515 31186 raft_consensus.cc:2749] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:31.172367 31184 raft_consensus.cc:2749] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:31.217546 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:31.220041 22107 tablet_replica.cc:333] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.221331 22107 raft_consensus.cc:2243] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.222496 22107 raft_consensus.cc:2272] T 2abf7c5535ea498fa1d3c949bcdc25ca P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.231698 22107 tablet_replica.cc:333] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.232621 22107 raft_consensus.cc:2243] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.244146 22107 raft_consensus.cc:2272] T 350816ae62fd4fb598a9ea6c84adf933 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.246878 22107 tablet_replica.cc:333] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.247745 22107 raft_consensus.cc:2243] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.272686 22107 raft_consensus.cc:2272] T d2bbbae982b14adea2da3b5b5217b18e P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.288168 31184 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:31.288812 31184 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.294552 31184 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.296038 31184 raft_consensus.cc:493] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:31.297722 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.300141 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8cb992d1f8470a8b883e9287064025" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.300496 31184 raft_consensus.cc:515] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:31.303875 31184 leader_election.cc:290] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:31.307991 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.308254 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d1af6a7ce14f5387824508ce04bfd0" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:31.310662 22107 tablet_replica.cc:333] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.311388 23287 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.311554 22107 raft_consensus.cc:2243] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.312974 23287 leader_election.cc:304] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.319099 22107 raft_consensus.cc:2272] T 2cd3243d8c2d447ca873f0890f2c0058 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.322111 31189 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:31.322988 31189 raft_consensus.cc:2749] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:31.331632 22107 tablet_replica.cc:333] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.332533 22107 raft_consensus.cc:2243] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.333112 22107 raft_consensus.cc:2272] T e3dffd4f7c72413b84a3862372dcce85 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.344734 31184 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:31.345487 31184 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:31.353361 31184 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:31.357039 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:31.358501 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "271ce3c0dde64506a60105c55526e4a3" candidate_uuid: "2778b449761a46708f2f643b1b5b5854" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:31.369763 23287 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2778b449761a46708f2f643b1b5b5854; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:31.371876 31184 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:31.376259 22107 tablet_replica.cc:333] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.377312 22107 raft_consensus.cc:2243] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.378450 22107 raft_consensus.cc:2272] T baab486079784de4933d5f96a0a5321b P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.381263 22107 tablet_replica.cc:333] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.406168 22107 raft_consensus.cc:2243] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.407337 22107 raft_consensus.cc:2272] T ec46374214e6444fa4c5afabe5bb309a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.413115 22107 tablet_replica.cc:333] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.413938 22107 raft_consensus.cc:2243] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.414510 22107 raft_consensus.cc:2272] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.421468 22107 tablet_replica.cc:333] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.422371 22107 raft_consensus.cc:2243] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.429839 22107 raft_consensus.cc:2272] T c5d8af85525845a192206b43a61f5c41 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.432859 22107 tablet_replica.cc:333] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.433713 22107 raft_consensus.cc:2243] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.434989 22107 raft_consensus.cc:2272] T 71b8a974ecc34bd487fe9c208e6acc17 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.437958 22107 tablet_replica.cc:333] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.438814 22107 raft_consensus.cc:2243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.439980 22107 raft_consensus.cc:2272] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.452212 22107 tablet_replica.cc:333] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.453107 22107 raft_consensus.cc:2243] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.477742 22107 raft_consensus.cc:2272] T b055a413d01c460895f3bc5034afc461 P 2778b449761a46708f2f643b1b5b5854 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.480579 22107 tablet_replica.cc:333] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.481614 22107 raft_consensus.cc:2243] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.482602 22107 raft_consensus.cc:2272] T 97dc7e2c632a45499346d6fcf8e4aee9 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.492169 22107 tablet_replica.cc:333] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.492997 22107 raft_consensus.cc:2243] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.517692 22107 raft_consensus.cc:2272] T f55bd3b311de43ba8aca6be22937f68a P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.520937 22107 tablet_replica.cc:333] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.521850 22107 raft_consensus.cc:2243] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.522907 22107 raft_consensus.cc:2272] T 6f8cb992d1f8470a8b883e9287064025 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.540946 22107 tablet_replica.cc:333] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.541888 22107 raft_consensus.cc:2243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.542588 22107 raft_consensus.cc:2272] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.578172 22107 tablet_replica.cc:333] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.597965 22107 raft_consensus.cc:2243] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.599381 22107 raft_consensus.cc:2272] T fb9a5906264b4cae9c8e888ad57c13c8 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.609714 22107 tablet_replica.cc:333] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.610626 22107 raft_consensus.cc:2243] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.611670 22107 raft_consensus.cc:2272] T 271ce3c0dde64506a60105c55526e4a3 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.618710 22107 tablet_replica.cc:333] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.619617 22107 raft_consensus.cc:2243] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.620759 22107 raft_consensus.cc:2272] T 446014759a9c4a0f9c99759951367083 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.623591 22107 tablet_replica.cc:333] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.632905 22107 raft_consensus.cc:2243] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.638793 22107 raft_consensus.cc:2272] T 1399c0b792a048cda2bcae40b23432c0 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.641532 22107 tablet_replica.cc:333] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.642356 22107 raft_consensus.cc:2243] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.642902 22107 raft_consensus.cc:2272] T ca56a943ba904b779d7584f94c266344 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.645612 22107 tablet_replica.cc:333] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.646323 22107 raft_consensus.cc:2243] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.646821 22107 raft_consensus.cc:2272] T 9689b2f60f934eb6b1c067507423b816 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.649412 22107 tablet_replica.cc:333] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.663630 22107 raft_consensus.cc:2243] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.664772 22107 raft_consensus.cc:2272] T dcbe5680eae64fa2a2565e9b0973703d P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.682796 22107 tablet_replica.cc:333] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.683589 22107 raft_consensus.cc:2243] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.684670 22107 raft_consensus.cc:2272] T 9d236b23181b45e5a16e91a3b6a30058 P 2778b449761a46708f2f643b1b5b5854 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.687291 22107 tablet_replica.cc:333] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.688081 22107 raft_consensus.cc:2243] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.688601 22107 raft_consensus.cc:2272] T 29bc5916fff44ea3b1dcd31091d822b9 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.713054 22107 tablet_replica.cc:333] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.713992 22107 raft_consensus.cc:2243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.715128 22107 raft_consensus.cc:2272] T cfaa0d3f687341a1ac860baa49e8bfb4 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.730435 22107 tablet_replica.cc:333] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.731266 22107 raft_consensus.cc:2243] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.731900 22107 raft_consensus.cc:2272] T ba09297a7e5c4b4f8b873c36c70e2a2d P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.734578 22107 tablet_replica.cc:333] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.738118 22107 raft_consensus.cc:2243] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.750653 22107 raft_consensus.cc:2272] T 32256a541626459187bbe0f5f7e6643c P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.753360 22107 tablet_replica.cc:333] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.754217 22107 raft_consensus.cc:2243] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.755084 22107 raft_consensus.cc:2272] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 2778b449761a46708f2f643b1b5b5854 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.757773 22107 tablet_replica.cc:333] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.758498 22107 raft_consensus.cc:2243] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.761140 22107 raft_consensus.cc:2272] T 87e270393fc841ddac0d9170b0bc6554 P 2778b449761a46708f2f643b1b5b5854 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.763818 22107 tablet_replica.cc:333] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.764564 22107 raft_consensus.cc:2243] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.774104 22107 raft_consensus.cc:2272] T 22fc2096e78f47158bbb4bae3a474e2c P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.777246 22107 tablet_replica.cc:333] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.777992 22107 raft_consensus.cc:2243] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.778481 22107 raft_consensus.cc:2272] T c3d1af6a7ce14f5387824508ce04bfd0 P 2778b449761a46708f2f643b1b5b5854 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.791158 22107 tablet_replica.cc:333] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.791947 22107 raft_consensus.cc:2243] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.792538 22107 raft_consensus.cc:2272] T 709006b08c044ed38e2df81795ed1613 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.795959 22107 tablet_replica.cc:333] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.796738 22107 raft_consensus.cc:2243] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.849960 22107 raft_consensus.cc:2272] T 03644bf5f16347c98fd9217f8a7fab79 P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.852715 22107 tablet_replica.cc:333] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.853521 22107 raft_consensus.cc:2243] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.854511 22107 raft_consensus.cc:2272] T 746c120ff6e048eaa7b3a03be2b8a91e P 2778b449761a46708f2f643b1b5b5854 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.870321 22107 tablet_replica.cc:333] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.871070 22107 raft_consensus.cc:2243] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.872179 22107 raft_consensus.cc:2272] T dbf1b9383e214ea7bb12a7f8784bffd1 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.874994 22107 tablet_replica.cc:333] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.875734 22107 raft_consensus.cc:2243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.876297 22107 raft_consensus.cc:2272] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 2778b449761a46708f2f643b1b5b5854 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.900419 22107 tablet_replica.cc:333] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.901279 22107 raft_consensus.cc:2243] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.901901 22107 raft_consensus.cc:2272] T 1bca0d75a0cb485ebd79c619c3bf8341 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.905061 22107 tablet_replica.cc:333] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.929932 22107 raft_consensus.cc:2243] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.931105 22107 raft_consensus.cc:2272] T c192a749705545f298d61bbb8ccb5665 P 2778b449761a46708f2f643b1b5b5854 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.933893 22107 tablet_replica.cc:333] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.934609 22107 raft_consensus.cc:2243] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.935253 22107 raft_consensus.cc:2272] T 06d137914eb14302bb888a191819fb75 P 2778b449761a46708f2f643b1b5b5854 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.956859 22107 tablet_replica.cc:333] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.957732 22107 raft_consensus.cc:2243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.958313 22107 raft_consensus.cc:2272] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 2778b449761a46708f2f643b1b5b5854 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.961015 22107 tablet_replica.cc:333] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:31.998732 22107 raft_consensus.cc:2243] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.999368 22107 raft_consensus.cc:2272] T f6161f3a92cf44138f9a68a6f005c4e9 P 2778b449761a46708f2f643b1b5b5854 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:32.002164 22107 tablet_replica.cc:333] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:32.002841 22107 raft_consensus.cc:2243] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:32.003401 22107 raft_consensus.cc:2272] T ea456647290e4959b61a6fa23ecfbfd4 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:32.023590 22107 tablet_replica.cc:333] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:32.024260 22107 raft_consensus.cc:2243] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:32.024974 22107 raft_consensus.cc:2272] T a59619d146f04895b0e204f6e4bdcf47 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:32.028340 22107 tablet_replica.cc:333] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854: stopping tablet replica
I20260709 12:16:32.028997 22107 raft_consensus.cc:2243] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:32.053706 22107 raft_consensus.cc:2272] T d736b5901fbd4de09b83cc09a8d00fd1 P 2778b449761a46708f2f643b1b5b5854 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:32.229480 23360 consensus_peers.cc:597] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.266885 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.326128 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.345469 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.348515 31190 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:32.350916 23360 consensus_peers.cc:597] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.353799 31175 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.357726 31175 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.350708 31190 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:32.363049 23204 catalog_manager.cc:4755] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): ChangeConfig:REMOVE_PEER RPC failed for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:32.361166 31175 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:32.370303 23466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.374797 31190 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:32.373672 23360 consensus_peers.cc:597] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.402341 23980 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:32.402843 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:32.405081 31190 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:32.431071 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:32.444271 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.448673 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.460238 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.465631 23360 consensus_peers.cc:597] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.480190 31089 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.480809 31089 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.483915 31089 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:32.497848 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.513901 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.518352 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:32.521211 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:32.522205 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.519901 23360 leader_election.cc:336] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:32.523161 31180 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.523768 31180 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.526424 31180 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.528173 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:32.528846 24012 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 16.
W20260709 12:16:32.529492 23434 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:32.530181 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:32.531347 31180 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:16:32.531785 31180 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.532230 31180 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:32.537307 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:32.538206 31190 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:32.540344 31180 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:32.545717 23434 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:32.549998 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "350816ae62fd4fb598a9ea6c84adf933" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:32.550573 24012 raft_consensus.cc:3060] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:16:32.551011 31180 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.558641 24012 raft_consensus.cc:2468] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:32.560019 23436 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:32.561353 31180 raft_consensus.cc:2804] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:16:32.566442 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.570138 31180 raft_consensus.cc:697] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:32.571066 31180 consensus_queue.cc:237] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.580000 23362 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:32.580817 31089 raft_consensus.cc:2749] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:32.591354 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.636132 23214 catalog_manager.cc:5697] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 16 to 17, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 17 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:32.648505 23360 consensus_peers.cc:597] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.649091 23360 consensus_peers.cc:597] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.658025 23360 consensus_peers.cc:597] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.658506 23360 consensus_peers.cc:597] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.658804 31089 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.659322 31089 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.661814 31219 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.661972 31190 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.662424 31219 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:32.661221 31089 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.666908 31175 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.667470 31175 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:32.675243 23360 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.662559 31190 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:32.678172 23360 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.678726 23360 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.680456 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.665233 31219 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:32.677249 31190 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:32.678731 31175 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:32.693660 31175 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.694214 31175 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.700580 31175 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:32.701859 23980 leader_election.cc:336] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.703663 23980 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.741159 31198 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.741817 31198 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.744767 31200 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.746027 31219 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.746903 31198 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.747526 31198 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
W20260709 12:16:32.748261 23980 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.748034 31198 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:32.751734 31198 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:32.746603 31219 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.745370 31200 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.753369 31175 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.759415 31219 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.776377 31175 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.779939 31175 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:32.760852 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.782210 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:32.759732 31200 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:32.768019 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.783519 23434 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.784159 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.768878 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.786041 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:32.786439 23396 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:32.786787 24012 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
I20260709 12:16:32.769568 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.788159 23392 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 21.
I20260709 12:16:32.788213 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:32.788951 24012 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 20.
W20260709 12:16:32.791791 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.794314 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.794299 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:32.795547 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 128719us
I20260709 12:16:32.797832 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.798573 23462 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:32.800367 31229 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.800993 31229 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.805145 31229 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:32.812407 23434 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.814436 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.815191 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 15.
I20260709 12:16:32.822595 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:32.823280 24010 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:32.824651 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:32.824844 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:32.830951 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 59449.2us
W20260709 12:16:32.835856 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.854110 23436 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:32.855028 31180 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:32.883524 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.885286 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:32.886827 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:32.888141 31190 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.888738 31190 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.889652 31180 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:32.890060 31180 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.890416 31180 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:32.890949 31190 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:32.887998 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.894696 23392 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
I20260709 12:16:32.898666 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.900324 31198 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:32.900610 31190 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.900789 31198 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.901268 31190 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.903628 31190 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:32.904107 23362 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:33909, user_credentials={real_user=slave}} blocked reactor thread for 130913us
I20260709 12:16:32.901275 31198 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:32.906003 31226 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.906251 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:32.906550 31226 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.908087 31219 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:32.908475 31219 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.908722 31226 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:32.908893 31219 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:32.916673 31198 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:32.921660 23980 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.922170 23980 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:32.924240 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.925045 31190 raft_consensus.cc:2749] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:32.925724 31190 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.926234 31190 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.929165 31190 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:32.930763 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:32.912009 31180 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.931396 23462 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:32.936304 31219 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:32.936376 23980 leader_election.cc:336] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.937984 23980 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.938462 23434 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:32.940397 23980 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:32.942543 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:32.943387 31228 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:32.943856 31228 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.944293 31228 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:32.944928 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:32.945542 24010 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:32.952879 24010 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:16:32.960302 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:32.944322 31180 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:32.966270 31198 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:32.975404 31219 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:32.978258 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:32.980077 23392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.980638 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.981356 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.982023 23396 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 17.
I20260709 12:16:32.980744 23392 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:32.982419 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.983258 23391 raft_consensus.cc:2393] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 25: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
I20260709 12:16:32.986428 31225 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:32.987182 31225 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:32.990926 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.991560 23391 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 20.
I20260709 12:16:32.995146 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:32.996120 23980 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.015275 23396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:32.997321 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:33.014546 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.023111 23396 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 15.
I20260709 12:16:33.002352 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
W20260709 12:16:32.997539 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 110439us
W20260709 12:16:33.024331 23360 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.024987 23360 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.025292 23360 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:33.028129 31226 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:32.997575 31225 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.008498 31219 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.031980 31219 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.034337 31200 raft_consensus.cc:2749] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:33.034531 31231 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:33.035175 31231 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.035609 31231 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:32.997217 31228 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.049556 31200 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.050196 31200 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.054430 31228 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.054683 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb9a5906264b4cae9c8e888ad57c13c8" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.057102 24010 raft_consensus.cc:3060] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:33.058051 31200 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.056298 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:33.056488 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:33.061076 31226 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:33.064077 24010 raft_consensus.cc:2468] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 25.
I20260709 12:16:33.065901 23982 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.067194 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.067643 31190 raft_consensus.cc:2764] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:16:33.067847 31175 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:33.066872 31225 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.068521 31175 raft_consensus.cc:493] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.068998 31175 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:33.068656 31225 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.071376 31225 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.075888 31231 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:33.077807 23980 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.078135 31231 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.075572 31219 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.083690 31175 raft_consensus.cc:515] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.086046 31175 leader_election.cc:290] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:33.082863 31219 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.086890 31219 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.088940 31219 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:33.090670 23980 leader_election.cc:336] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.093231 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:33.094966 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec46374214e6444fa4c5afabe5bb309a" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.095579 24015 raft_consensus.cc:3060] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:33.097761 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:33.098496 23462 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6253b834ffb744579b46fcede2fc0e91 in current term 22: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
I20260709 12:16:33.100397 31190 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:33.102255 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.104254 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.104332 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.104811 24012 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 18.
I20260709 12:16:33.105003 24010 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 23.
I20260709 12:16:33.107230 24015 raft_consensus.cc:2468] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 21.
I20260709 12:16:33.111268 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.112506 31190 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:16:33.112732 23982 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.112974 31190 raft_consensus.cc:493] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.113487 31190 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:33.114765 31175 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:16:33.115193 31175 raft_consensus.cc:493] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.115563 31175 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:33.118096 23362 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
W20260709 12:16:33.113962 23360 consensus_peers.cc:597] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:33.119634 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:33.114056 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:33.125118 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:33.126355 31231 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:33.126758 31231 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.127115 31231 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:33.127187 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:33.131716 31200 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:16:33.131791 31225 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:33.132194 31200 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.132660 31200 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:33.124677 31228 raft_consensus.cc:2804] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:16:33.133929 31228 raft_consensus.cc:697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:33.134778 31228 consensus_queue.cc:237] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.136757 31198 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.137157 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b87a4fcda7f4296b7f91f6ba3a14f29" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.137806 23391 raft_consensus.cc:3060] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:33.138150 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.138844 23388 raft_consensus.cc:2393] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 22: Already voted for candidate 6253b834ffb744579b46fcede2fc0e91 in this term.
W20260709 12:16:33.140026 23360 consensus_peers.cc:597] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:33.137290 31198 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.140484 23360 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.145507 23360 leader_election.cc:336] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.145907 23360 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:33.146222 23391 raft_consensus.cc:2468] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:16:33.151674 23434 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.152282 31190 raft_consensus.cc:515] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.160784 23362 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:33.153280 31219 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:33.153061 31198 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.163120 31089 raft_consensus.cc:2804] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:33.164299 31089 raft_consensus.cc:697] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:33.165126 31089 consensus_queue.cc:237] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.168164 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:33.172556 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "baab486079784de4933d5f96a0a5321b" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.173346 23391 raft_consensus.cc:3060] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:33.161715 31229 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.186175 31175 raft_consensus.cc:515] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.193140 23980 leader_election.cc:336] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.174088 31198 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:33.198621 23391 raft_consensus.cc:2468] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:33.194556 31229 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.208973 31229 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.211133 31231 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.213286 31190 leader_election.cc:290] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:33.213744 23434 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.215112 23980 leader_election.cc:336] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.223776 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.224448 24012 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:33.234287 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:33.235344 31235 log.cc:927] Time spent T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:16:33.238546 31200 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.240810 31200 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.245676 31226 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:33.246979 31175 leader_election.cc:290] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.248531 31231 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.249559 23218 catalog_manager.cc:5697] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 24 to 25, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 25 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.257614 31175 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.279232 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
W20260709 12:16:33.279633 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 94520.3us
I20260709 12:16:33.279893 23463 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:33.280802 23436 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.283100 23982 leader_election.cc:304] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.286033 31238 raft_consensus.cc:2804] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:33.286550 31238 raft_consensus.cc:697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:33.287396 31238 consensus_queue.cc:237] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.289280 23218 catalog_manager.cc:5697] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 20 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 21 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:33.294718 23360 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.295915 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446014759a9c4a0f9c99759951367083" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:16:33.296028 23360 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.297103 23360 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:33.297379 24015 raft_consensus.cc:1275] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:16:33.297951 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87e270393fc841ddac0d9170b0bc6554" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.298393 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.298547 24010 raft_consensus.cc:3060] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:16:33.298997 24016 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:33.296530 24012 raft_consensus.cc:3060] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:33.284451 31236 raft_consensus.cc:2804] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:33.271711 31175 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.306330 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d236b23181b45e5a16e91a3b6a30058" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.306967 23395 raft_consensus.cc:3060] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:16:33.314139 23395 raft_consensus.cc:2468] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:33.315434 23391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71b8a974ecc34bd487fe9c208e6acc17" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.316000 23391 raft_consensus.cc:3060] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:16:33.317013 23463 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:33.317265 31236 raft_consensus.cc:697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:33.318317 31236 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.318899 24010 raft_consensus.cc:2468] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 24.
W20260709 12:16:33.322007 23360 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.322554 31200 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:16:33.296507 31198 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.315057 31175 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:33.323529 24012 raft_consensus.cc:2468] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 19.
I20260709 12:16:33.323161 31198 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.324698 23362 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:33.325508 31198 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.325698 31234 raft_consensus.cc:2804] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:16:33.326181 31234 raft_consensus.cc:697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:33.327095 31234 consensus_queue.cc:237] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.332406 23391 raft_consensus.cc:2468] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 16.
I20260709 12:16:33.333843 23362 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 
I20260709 12:16:33.341593 31239 raft_consensus.cc:2804] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:33.342142 31239 raft_consensus.cc:697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:33.342981 31239 consensus_queue.cc:237] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.347512 23980 leader_election.cc:336] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.352934 23360 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.357474 23434 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.357124 23360 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.358615 23360 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.416555 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91" is_pre_election: true
I20260709 12:16:33.417189 23395 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 21.
I20260709 12:16:33.389566 31242 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.421986 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.424883 23218 catalog_manager.cc:5697] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.433498 31180 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:33.433990 31180 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.434368 31180 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:33.436594 31247 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.422433 31242 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.439451 23214 catalog_manager.cc:5697] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 23 to 24, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 24 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.441526 23214 catalog_manager.cc:5697] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 18 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 19 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.454504 31242 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:33.459899 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.466131 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.466328 31240 consensus_queue.cc:1048] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:16:33.469606 31198 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:16:33.470026 31198 raft_consensus.cc:493] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.470378 31198 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:16:33.437143 31247 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.475256 31247 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.478420 23982 leader_election.cc:304] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.479653 31226 raft_consensus.cc:2804] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:16:33.480161 31226 raft_consensus.cc:697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:33.480602 23982 leader_election.cc:304] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.482658 31247 raft_consensus.cc:2804] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:16:33.483140 31247 raft_consensus.cc:697] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:33.483955 31180 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.484115 31247 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.490757 23980 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.489712 31226 consensus_queue.cc:237] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.500587 31244 log.cc:927] Time spent T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: Append to log took a long time: real 0.093s	user 0.002s	sys 0.000s
I20260709 12:16:33.493420 31180 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.502101 31198 raft_consensus.cc:515] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.503609 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.504304 31198 leader_election.cc:290] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.504457 31225 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:16:33.504940 31225 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.505309 31225 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:16:33.536154 23360 consensus_peers.cc:597] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.538472 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:33.539106 23463 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 11.
I20260709 12:16:33.540505 24011 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.540923 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:33.541117 24011 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 11.
I20260709 12:16:33.543738 24010 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
I20260709 12:16:33.543021 31225 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.553058 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32256a541626459187bbe0f5f7e6643c" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.553797 24014 raft_consensus.cc:3060] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:16:33.559340 23980 leader_election.cc:336] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.563675 24014 raft_consensus.cc:2468] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
W20260709 12:16:33.564023 23434 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.571712 23434 leader_election.cc:336] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.575928 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cfaa0d3f687341a1ac860baa49e8bfb4" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.576694 24013 raft_consensus.cc:3060] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:16:33.584264 23204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 73641.3us
I20260709 12:16:33.586906 24013 raft_consensus.cc:2468] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 20.
I20260709 12:16:33.588250 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.588770 31225 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:33.591570 31198 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:33.591969 31198 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.592154 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.592625 31198 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:16:33.594708 23436 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.595305 31180 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:33.596266 31236 raft_consensus.cc:2804] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:33.596626 31180 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.596794 31236 raft_consensus.cc:697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:33.597205 31180 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:33.597828 31236 consensus_queue.cc:237] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.602015 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.605044 23214 catalog_manager.cc:5697] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 20 to 21, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 21 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.605122 31175 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:16:33.609371 31175 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 2778b449761a46708f2f643b1b5b5854)
I20260709 12:16:33.609802 31175 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:16:33.611480 22107 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 23342)
I20260709 12:16:33.620115 31198 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.636596 23204 catalog_manager.cc:4755] TS 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): ChangeConfig:REMOVE_PEER RPC failed for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.640378 31180 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.641685 23214 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 18, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.649689 23395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6253b834ffb744579b46fcede2fc0e91"
I20260709 12:16:33.650287 23395 raft_consensus.cc:3060] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:16:33.652352 23436 leader_election.cc:304] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.653200 31236 raft_consensus.cc:2804] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:16:33.657106 23395 raft_consensus.cc:2468] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 22.
I20260709 12:16:33.661413 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "746c120ff6e048eaa7b3a03be2b8a91e" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.662005 24013 raft_consensus.cc:3060] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:33.662451 24014 raft_consensus.cc:1275] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:16:33.664053 31231 consensus_queue.cc:1048] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:16:33.664328 31175 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.669906 31198 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.645411 31180 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:33.672043 23982 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.678433 24013 raft_consensus.cc:2468] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:33.679342 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:33.679968 24010 raft_consensus.cc:2393] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 12: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
I20260709 12:16:33.653750 31236 raft_consensus.cc:697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:33.679373 31225 raft_consensus.cc:2804] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:16:33.683135 31225 raft_consensus.cc:697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Becoming Leader. State: Replica: 54640ab8abeb4c949426d66c7e5aed87, State: Running, Role: LEADER
I20260709 12:16:33.684052 31225 consensus_queue.cc:237] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:33.690830 31252 log.cc:927] Time spent T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:16:33.694357 23434 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.694991 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:33.697391 23980 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:33.702719 23436 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:33.703927 31180 raft_consensus.cc:2804] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:33.704459 31180 raft_consensus.cc:697] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:33.705252 31180 consensus_queue.cc:237] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.708576 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:33.709355 31198 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:16:33.745920 31236 consensus_queue.cc:237] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:33.749171 31175 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:33.771710 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:33.772553 23463 raft_consensus.cc:2393] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 12: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:33.782790 23360 consensus_peers.cc:597] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.784480 23982 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:33.871025 24010 raft_consensus.cc:1275] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:16:33.874032 31260 raft_consensus.cc:2749] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:16:33.900192 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:33.901257 23360 consensus_peers.cc:597] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.916860 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 15 to 16, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 16 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.919510 23214 catalog_manager.cc:5697] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: term changed from 21 to 22, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196). New cstate: current_term: 22 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:33.930332 23360 consensus_peers.cc:597] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.939345 24014 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:16:33.939625 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.946199 31229 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:16:33.949271 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:33.951923 24015 raft_consensus.cc:1275] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:16:33.952837 23395 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:16:33.963776 31236 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:16:33.965502 23214 catalog_manager.cc:5697] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 19, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 19 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.967250 23214 catalog_manager.cc:5697] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 19 to 20, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 20 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.970220 24015 raft_consensus.cc:1275] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 20. (index mismatch)
I20260709 12:16:33.977365 23395 raft_consensus.cc:1275] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:16:33.984488 23388 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:16:33.984741 31239 consensus_queue.cc:1048] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:16:33.986035 23395 raft_consensus.cc:1275] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:16:33.986603 31234 consensus_queue.cc:1048] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:33.993642 31241 consensus_queue.cc:1048] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
I20260709 12:16:34.002888 23218 catalog_manager.cc:5697] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 18, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.012259 31226 consensus_queue.cc:1048] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:16:34.015275 31260 consensus_queue.cc:1048] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:16:34.024701 31261 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:16:34.123374 23466 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:16:34.140667 24013 raft_consensus.cc:1275] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:16:34.151187 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.152166 23434 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.156455 31236 consensus_queue.cc:1048] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:16:34.144517 31261 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:16:34.177501 23393 raft_consensus.cc:1275] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
W20260709 12:16:34.196762 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.208829 31261 consensus_queue.cc:1048] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:16:34.278882 24013 raft_consensus.cc:1275] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
W20260709 12:16:34.292734 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.315028 23434 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.315830 31180 consensus_queue.cc:1048] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:16:34.376354 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 50823.5us
I20260709 12:16:34.387166 24009 raft_consensus.cc:1275] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 12:16:34.484128 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 102034us
I20260709 12:16:34.499234 31240 consensus_queue.cc:1048] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:16:34.583909 23360 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 65227.4us
W20260709 12:16:34.612038 22107 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 23342)
I20260709 12:16:34.647908 24010 consensus_queue.cc:237] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:34.657047 23396 raft_consensus.cc:1275] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Refusing update from remote peer 54640ab8abeb4c949426d66c7e5aed87: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 12:16:34.664785 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.671692 31226 consensus_queue.cc:1048] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:16:34.674443 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:34.682394 31175 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Committing config change with OpId 16.15: config changed from index 13 to 15, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:34.696868 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 71b8a974ecc34bd487fe9c208e6acc17 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:16:34.699908 23391 raft_consensus.cc:2955] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Committing config change with OpId 16.15: config changed from index 13 to 15, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:34.705556 23214 catalog_manager.cc:5697] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 13 to 15, VOTER da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195) evicted. New cstate: current_term: 16 leader_uuid: "54640ab8abeb4c949426d66c7e5aed87" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:16:34.850275 23360 consensus_peers.cc:597] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:34.856173 23457 tablet_service.cc:1558] Processing DeleteTablet for tablet 71b8a974ecc34bd487fe9c208e6acc17 with delete_type TABLET_DATA_TOMBSTONED (TS da1e2fd157894634bf92d6c82c06ec64 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:43504
W20260709 12:16:34.882524 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:34.886879 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:34.890925 23360 consensus_peers.cc:597] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:34.900532 31294 tablet_replica.cc:333] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:34.901418 31294 raft_consensus.cc:2243] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.902295 31294 raft_consensus.cc:2272] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.919600 31294 ts_tablet_manager.cc:1916] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:16:34.959755 23434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111) [suppressed 397 similar messages]
I20260709 12:16:35.017696 31241 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:35.029182 23360 consensus_peers.cc:597] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.029112 31241 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:35.034844 31294 ts_tablet_manager.cc:1929] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.14
I20260709 12:16:35.035284 31294 log.cc:1199] T 71b8a974ecc34bd487fe9c208e6acc17 P da1e2fd157894634bf92d6c82c06ec64: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/wals/71b8a974ecc34bd487fe9c208e6acc17
I20260709 12:16:35.035585 31241 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:35.036531 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:35.037139 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 22.
I20260709 12:16:35.038339 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:35.039149 31241 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:16:35.039531 31241 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.039889 31241 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:35.045615 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.050422 23360 consensus_peers.cc:597] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.052716 23206 catalog_manager.cc:5028] TS da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): tablet 71b8a974ecc34bd487fe9c208e6acc17 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
I20260709 12:16:35.050901 31241 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:35.050982 23360 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:35.057948 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2abf7c5535ea498fa1d3c949bcdc25ca" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:35.058473 23462 raft_consensus.cc:3060] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:16:35.059204 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.068210 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.071229 23462 raft_consensus.cc:2468] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6253b834ffb744579b46fcede2fc0e91 in term 23.
I20260709 12:16:35.072932 23362 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:35.074121 31234 raft_consensus.cc:2804] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:16:35.075659 23360 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:35.077526 31241 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:35.087749 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.089463 23360 consensus_peers.cc:597] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.092468 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.102332 31234 raft_consensus.cc:697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Becoming Leader. State: Replica: 6253b834ffb744579b46fcede2fc0e91, State: Running, Role: LEADER
I20260709 12:16:35.114387 31234 consensus_queue.cc:237] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:35.127327 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.151700 23360 consensus_peers.cc:597] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.157559 23360 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.168485 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.181855 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.193392 23211 catalog_manager.cc:5697] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 reported cstate change: term changed from 21 to 23, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194). New cstate: current_term: 23 leader_uuid: "6253b834ffb744579b46fcede2fc0e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:35.210376 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.278439 31229 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.279035 31229 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:35.281183 31229 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:35.285774 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:35.286453 24013 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 12.
W20260709 12:16:35.289391 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:35.289999 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:35.292681 31240 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:16:35.293057 31240 raft_consensus.cc:493] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.293412 31240 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:16:35.301582 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:35.330763 31240 raft_consensus.cc:515] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:35.333033 31240 leader_election.cc:290] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Requested vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:35.337949 23434 leader_election.cc:336] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:35.338094 23360 consensus_peers.cc:597] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.344345 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c192a749705545f298d61bbb8ccb5665" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:35.344997 24013 raft_consensus.cc:3060] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:16:35.354858 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.380069 24013 raft_consensus.cc:2468] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 13.
W20260709 12:16:35.380751 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.381791 23436 leader_election.cc:304] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854
I20260709 12:16:35.383839 31180 raft_consensus.cc:2804] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:16:35.384271 31180 raft_consensus.cc:697] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:35.385054 31180 consensus_queue.cc:237] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:35.418836 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.438258 23211 catalog_manager.cc:5697] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 11 to 13, leader changed from 2778b449761a46708f2f643b1b5b5854 (127.21.150.193) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 13 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:35.462679 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.462960 23360 consensus_peers.cc:597] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.466118 23360 consensus_peers.cc:597] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.475215 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.538015 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.604220 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:35.612627 22107 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 23342)
W20260709 12:16:35.619331 23360 consensus_peers.cc:597] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.623704 23360 consensus_peers.cc:597] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:35.646274 23462 raft_consensus.cc:1275] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Refusing update from remote peer 6253b834ffb744579b46fcede2fc0e91: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:16:35.648104 31234 consensus_queue.cc:1048] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:16:35.733330 23466 consensus_queue.cc:237] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:35.767846 24013 raft_consensus.cc:1275] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:16:35.769973 31180 consensus_queue.cc:1048] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:16:35.777709 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.797241 23434 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:35.801450 31240 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 12 to 14, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:35.808676 24010 raft_consensus.cc:2955] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 12 to 14, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:35.841828 23211 catalog_manager.cc:5697] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 reported cstate change: config changed from index 12 to 14, VOTER 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) evicted. New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } } }
I20260709 12:16:35.849637 23206 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:16:35.856106 23380 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 with delete_type TABLET_DATA_TOMBSTONED (TS 6253b834ffb744579b46fcede2fc0e91 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:33622
W20260709 12:16:35.870810 23360 consensus_peers.cc:597] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:35.873581 24010 raft_consensus.cc:1275] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:16:35.883527 31240 consensus_queue.cc:1048] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:16:35.907855 31304 tablet_replica.cc:333] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:35.917662 31304 raft_consensus.cc:2243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:35.918551 31304 raft_consensus.cc:2272] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:35.931113 31304 ts_tablet_manager.cc:1916] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:16:35.949961 31304 ts_tablet_manager.cc:1929] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 12:16:35.950407 31304 log.cc:1199] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 6253b834ffb744579b46fcede2fc0e91: Deleting WAL directory at /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/wals/8b87a4fcda7f4296b7f91f6ba3a14f29
I20260709 12:16:35.952812 23206 catalog_manager.cc:5028] TS 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): tablet 8b87a4fcda7f4296b7f91f6ba3a14f29 (table test-workload [id=a60af847ced64e03baaf6f12357ca46d]) successfully deleted
W20260709 12:16:36.363907 23360 consensus_peers.cc:597] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.364915 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.372793 23434 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.375039 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 45365.2us
W20260709 12:16:36.386143 23360 consensus_peers.cc:597] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.400669 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.616567 23360 consensus_peers.cc:597] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.698037 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.711041 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.902626 23434 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:36.986421 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.310608 23360 consensus_peers.cc:597] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.352887 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.369150 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.431846 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.452876 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.456455 23360 consensus_peers.cc:597] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.478677 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:37.521624 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
W20260709 12:16:37.584640 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.587566 23360 consensus_peers.cc:597] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.590822 23360 consensus_peers.cc:597] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.595331 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.631531 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.634132 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.644817 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.672050 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.719806 23360 consensus_peers.cc:597] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.732302 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.771075 23360 consensus_peers.cc:597] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.779204 23360 consensus_peers.cc:597] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.808940 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:37.820508 23360 consensus_peers.cc:597] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.838883 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.839913 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.850055 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.886221 23360 consensus_peers.cc:597] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:37.895221 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:16:37.911609 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.935330 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.937165 23982 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.949905 23360 consensus_peers.cc:597] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:37.964027 23436 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.978574 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.985512 23982 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.007369 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:38.054019 23982 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.071493 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.083835 23436 consensus_peers.cc:597] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.091703 23436 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.072186 23982 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.116519 23436 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.125484 23436 consensus_peers.cc:597] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.134379 23982 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.134568 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:38.135825 22107 tablet_replica.cc:333] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.136955 22107 raft_consensus.cc:2243] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.138432 22107 raft_consensus.cc:2272] T fa5f71c212664b78a4deb373bf5ed2f5 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:38.139660 23982 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.141235 22107 tablet_replica.cc:333] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.146456 22107 raft_consensus.cc:2243] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.156666 22107 raft_consensus.cc:2272] T d2bbbae982b14adea2da3b5b5217b18e P 6253b834ffb744579b46fcede2fc0e91 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.160140 22107 tablet_replica.cc:333] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.161151 22107 raft_consensus.cc:2243] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 23 LEADER]: Raft consensus shutting down.
W20260709 12:16:38.162143 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:38.162515 22107 raft_consensus.cc:2272] T 2abf7c5535ea498fa1d3c949bcdc25ca P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.165925 22107 tablet_replica.cc:333] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
W20260709 12:16:38.171800 23436 consensus_peers.cc:597] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.170298 22107 raft_consensus.cc:2243] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Raft consensus shutting down.
W20260709 12:16:38.182643 23436 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.185743 22107 raft_consensus.cc:2272] T ec46374214e6444fa4c5afabe5bb309a P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.193548 22107 tablet_replica.cc:333] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.194882 22107 raft_consensus.cc:2243] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.195667 22107 raft_consensus.cc:2272] T baab486079784de4933d5f96a0a5321b P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:38.197492 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:38.199177 22107 tablet_replica.cc:333] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
W20260709 12:16:38.209254 23982 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.213961 22107 raft_consensus.cc:2243] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.215332 22107 raft_consensus.cc:2272] T 2cd3243d8c2d447ca873f0890f2c0058 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.218654 22107 tablet_replica.cc:333] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.219612 22107 raft_consensus.cc:2243] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.220451 22107 raft_consensus.cc:2272] T e3dffd4f7c72413b84a3862372dcce85 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.223368 22107 tablet_replica.cc:333] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
W20260709 12:16:38.252542 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.295655 23360 consensus_peers.cc:597] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:38.321460 23436 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.334221 23982 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.341212 22107 raft_consensus.cc:2243] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.345022 22107 raft_consensus.cc:2272] T abd134c48fc14e60af68e7bff27733c0 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:38.336097 23360 consensus_peers.cc:597] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:38.375555 23436 consensus_peers.cc:597] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.391011 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:38.407044 22107 tablet_replica.cc:333] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.412062 22107 raft_consensus.cc:2243] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.415568 22107 raft_consensus.cc:2272] T 5e2145c1046c436e8ca4a54ef0df4ab5 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:38.408609 23436 consensus_peers.cc:597] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:38.410842 23982 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.434135 22107 tablet_replica.cc:333] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
W20260709 12:16:38.452668 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:38.481693 22107 raft_consensus.cc:2243] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.530798 22107 raft_consensus.cc:2272] T 19d9facb5378417aaa3279b3dd9f02a9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.534372 22107 tablet_replica.cc:333] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.541076 22107 raft_consensus.cc:2243] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 LEADER]: Raft consensus shutting down.
W20260709 12:16:38.567370 23434 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:38.589355 22107 raft_consensus.cc:2272] T f55bd3b311de43ba8aca6be22937f68a P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.592664 22107 tablet_replica.cc:333] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.593573 22107 raft_consensus.cc:2243] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.618093 22107 raft_consensus.cc:2272] T 6f8cb992d1f8470a8b883e9287064025 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.628444 22107 tablet_replica.cc:333] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.641700 22107 raft_consensus.cc:2243] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.651574 22107 raft_consensus.cc:2272] T 71b8a974ecc34bd487fe9c208e6acc17 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.660172 22107 tablet_replica.cc:333] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.661072 22107 raft_consensus.cc:2243] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.661865 22107 raft_consensus.cc:2272] T c5d8af85525845a192206b43a61f5c41 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.665268 22107 tablet_replica.cc:333] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.678939 22107 raft_consensus.cc:2243] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.691442 22107 raft_consensus.cc:2272] T de7238eb389343cd954cb756bed5925f P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.696427 22107 tablet_replica.cc:333] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.697337 22107 raft_consensus.cc:2243] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:38.697755 23360 consensus_peers.cc:597] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:38.700053 22107 raft_consensus.cc:2272] T ba09297a7e5c4b4f8b873c36c70e2a2d P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.703241 22107 tablet_replica.cc:333] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
W20260709 12:16:38.706213 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:38.755198 22107 raft_consensus.cc:2243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.756069 22107 raft_consensus.cc:2272] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.761566 22107 tablet_replica.cc:333] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.762404 22107 raft_consensus.cc:2243] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.763610 22107 raft_consensus.cc:2272] T fb9a5906264b4cae9c8e888ad57c13c8 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.772867 22107 tablet_replica.cc:333] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.776108 22107 raft_consensus.cc:2243] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.784353 22107 raft_consensus.cc:2272] T 271ce3c0dde64506a60105c55526e4a3 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.792608 22107 tablet_replica.cc:333] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.793552 22107 raft_consensus.cc:2243] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.794735 22107 raft_consensus.cc:2272] T 97dc7e2c632a45499346d6fcf8e4aee9 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.808905 22107 tablet_replica.cc:333] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.809849 22107 raft_consensus.cc:2243] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.812922 22107 raft_consensus.cc:2272] T ca56a943ba904b779d7584f94c266344 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.821792 22107 tablet_replica.cc:333] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.826488 22107 raft_consensus.cc:2243] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.827428 22107 raft_consensus.cc:2272] T a658e02588314c9ca8a73f5eb423fb27 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.832316 22107 tablet_replica.cc:333] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.838004 22107 raft_consensus.cc:2243] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.839370 22107 raft_consensus.cc:2272] T 446014759a9c4a0f9c99759951367083 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.854447 22107 tablet_replica.cc:333] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.858136 22107 raft_consensus.cc:2243] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.862452 22107 raft_consensus.cc:2272] T 9d236b23181b45e5a16e91a3b6a30058 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.879946 22107 tablet_replica.cc:333] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.886477 22107 raft_consensus.cc:2243] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.889920 22107 raft_consensus.cc:2272] T dcbe5680eae64fa2a2565e9b0973703d P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.893013 22107 tablet_replica.cc:333] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.897318 22107 raft_consensus.cc:2243] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.898752 22107 raft_consensus.cc:2272] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P 6253b834ffb744579b46fcede2fc0e91 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.912917 22107 tablet_replica.cc:333] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.919517 22107 raft_consensus.cc:2243] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:38.921250 22107 raft_consensus.cc:2272] T 87e270393fc841ddac0d9170b0bc6554 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.931581 22107 tablet_replica.cc:333] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:38.967842 22107 raft_consensus.cc:2243] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:38.971349 22107 raft_consensus.cc:2272] T 22fc2096e78f47158bbb4bae3a474e2c P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:38.980343 31319 raft_consensus.cc:493] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:38.981024 31319 raft_consensus.cc:515] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:38.984484 31319 leader_election.cc:290] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:38.986182 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:38.986318 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4de7d7863704646b55025720cc0da9c" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:38.993600 31319 raft_consensus.cc:493] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:38.994206 31319 raft_consensus.cc:515] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:38.996189 31319 leader_election.cc:290] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:38.995024 23362 leader_election.cc:304] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:38.997347 31319 raft_consensus.cc:2749] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:39.000382 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:39.004328 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2349832eeb6f4824800ff39a557daeb6" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.006528 23362 leader_election.cc:304] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:39.007759 31319 raft_consensus.cc:2749] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:38.985172 22107 tablet_replica.cc:333] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.020309 31319 raft_consensus.cc:493] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:39.021255 22107 raft_consensus.cc:2243] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.020942 31319 raft_consensus.cc:515] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.022037 22107 raft_consensus.cc:2272] T 6d83783f67774af1b007547d0aca8da1 P 6253b834ffb744579b46fcede2fc0e91 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.023348 31319 leader_election.cc:290] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.024432 31319 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:39.025053 31319 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.025624 22107 tablet_replica.cc:333] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.027426 31319 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.029942 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:39.030275 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:39.031294 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff06250da5d340babc2a28fd336380a7" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.033733 31319 raft_consensus.cc:493] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:39.034403 31319 raft_consensus.cc:515] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.034415 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b150e0c7494c95beb04642f466f450" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.037648 31319 leader_election.cc:290] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:39.041989 23360 leader_election.cc:336] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.045086 22107 raft_consensus.cc:2243] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.047175 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c7ea21ce0804c45ac69bf6430bcfb45" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.047791 31319 raft_consensus.cc:493] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:39.048419 31319 raft_consensus.cc:515] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.051577 31319 leader_election.cc:290] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:39.054368 23360 leader_election.cc:336] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.053118 22107 raft_consensus.cc:2272] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.049238 23362 leader_election.cc:304] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:39.059911 23362 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:39.060572 31319 raft_consensus.cc:2749] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:39.061278 23362 raft_consensus.cc:2686] T 57b150e0c7494c95beb04642f466f450 P 6253b834ffb744579b46fcede2fc0e91: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:16:39.062729 23362 leader_election.cc:304] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:39.069903 31319 raft_consensus.cc:2749] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:39.078436 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbf1b9383e214ea7bb12a7f8784bffd1" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.084939 23362 leader_election.cc:304] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:39.086614 31308 raft_consensus.cc:2749] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:39.088819 22107 tablet_replica.cc:333] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.092893 22107 raft_consensus.cc:2243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.099895 22107 raft_consensus.cc:2272] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.110411 22107 tablet_replica.cc:333] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.111337 22107 raft_consensus.cc:2243] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.113070 22107 raft_consensus.cc:2272] T 709006b08c044ed38e2df81795ed1613 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.115334 31308 raft_consensus.cc:493] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 54640ab8abeb4c949426d66c7e5aed87)
I20260709 12:16:39.127185 31308 raft_consensus.cc:515] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.129607 31308 leader_election.cc:290] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.132340 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bca0d75a0cb485ebd79c619c3bf8341" candidate_uuid: "6253b834ffb744579b46fcede2fc0e91" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.134117 22107 tablet_replica.cc:333] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.134985 22107 raft_consensus.cc:2243] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.135821 22107 raft_consensus.cc:2272] T 2349832eeb6f4824800ff39a557daeb6 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:39.136125 23360 leader_election.cc:336] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.136754 23360 leader_election.cc:304] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6253b834ffb744579b46fcede2fc0e91; no voters: 2778b449761a46708f2f643b1b5b5854, 54640ab8abeb4c949426d66c7e5aed87
I20260709 12:16:39.137537 31308 raft_consensus.cc:2749] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:39.138728 22107 tablet_replica.cc:333] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.139360 22107 raft_consensus.cc:2243] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.139987 22107 raft_consensus.cc:2272] T dbf1b9383e214ea7bb12a7f8784bffd1 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.142125 22107 tablet_replica.cc:333] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.142835 22107 raft_consensus.cc:2243] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.143698 22107 raft_consensus.cc:2272] T 1bca0d75a0cb485ebd79c619c3bf8341 P 6253b834ffb744579b46fcede2fc0e91 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.146661 22107 tablet_replica.cc:333] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.147526 22107 raft_consensus.cc:2243] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:39.148734 22107 raft_consensus.cc:2272] T 06d137914eb14302bb888a191819fb75 P 6253b834ffb744579b46fcede2fc0e91 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.151592 22107 tablet_replica.cc:333] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.155577 22107 raft_consensus.cc:2243] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:39.158263 22107 raft_consensus.cc:2272] T f6161f3a92cf44138f9a68a6f005c4e9 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.165331 22107 tablet_replica.cc:333] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.166183 22107 raft_consensus.cc:2243] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:39.167387 22107 raft_consensus.cc:2272] T 487ee45f93ab48478b7fa939ccf7f658 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.170419 22107 tablet_replica.cc:333] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.171207 22107 raft_consensus.cc:2243] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.172001 22107 raft_consensus.cc:2272] T 19b18a93b6e8478e9f44ce88b342ab54 P 6253b834ffb744579b46fcede2fc0e91 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.174623 22107 tablet_replica.cc:333] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.175396 22107 raft_consensus.cc:2243] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:39.176566 22107 raft_consensus.cc:2272] T 2984a5ac8b6b49c9aa8eb969a484d610 P 6253b834ffb744579b46fcede2fc0e91 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.179416 22107 tablet_replica.cc:333] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.180271 22107 raft_consensus.cc:2243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.181195 22107 raft_consensus.cc:2272] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 6253b834ffb744579b46fcede2fc0e91 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.183967 22107 tablet_replica.cc:333] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.184748 22107 raft_consensus.cc:2243] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.185570 22107 raft_consensus.cc:2272] T c4de7d7863704646b55025720cc0da9c P 6253b834ffb744579b46fcede2fc0e91 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.188333 22107 tablet_replica.cc:333] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.189071 22107 raft_consensus.cc:2243] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:39.190196 22107 raft_consensus.cc:2272] T ff06250da5d340babc2a28fd336380a7 P 6253b834ffb744579b46fcede2fc0e91 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.192947 22107 tablet_replica.cc:333] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91: stopping tablet replica
I20260709 12:16:39.193668 22107 raft_consensus.cc:2243] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:39.194902 22107 raft_consensus.cc:2272] T a59619d146f04895b0e204f6e4bdcf47 P 6253b834ffb744579b46fcede2fc0e91 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:39.323382 31318 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.324848 31318 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:39.327261 31318 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:39.335976 23436 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:39.367285 31298 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.368027 31298 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.380005 31298 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:39.382441 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.405668 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:39.506604 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 100990us
W20260709 12:16:39.508440 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:39.483238 31320 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.484738 31298 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.509874 31320 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.509991 31298 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.512326 31320 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:39.512449 31298 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:39.489624 31321 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.514189 31321 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:39.526630 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:39.527840 23982 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:41721, user_credentials={real_user=slave}} blocked reactor thread for 43893.2us
I20260709 12:16:39.527851 31313 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:39.527429 23436 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:39.529642 31321 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:39.545850 31240 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.546391 31240 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:39.548357 31240 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:39.558487 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:39.560842 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:39.567495 23982 connection.cc:769] client connection to 127.21.150.194:41721 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:16:39.570344 31321 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.570866 31321 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.572360 31321 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.574281 31321 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.574810 31321 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:39.576318 23436 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.576833 31321 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.580199 31316 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.580744 31316 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:39.581990 23982 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:39.582762 23436 leader_election.cc:336] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.583002 31316 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:39.582439 23982 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:39.585578 31298 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:39.586124 23436 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:39.585642 23982 leader_election.cc:336] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.587196 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:39.590689 31298 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:39.592099 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.592778 24014 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
I20260709 12:16:39.592144 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:39.594175 24010 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 14.
W20260709 12:16:39.594575 23436 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.596035 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.597615 31321 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:16:39.598017 31321 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.599298 31321 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:39.598724 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.601943 31316 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:16:39.602353 31316 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.602739 31316 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:39.648348 31316 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.655089 31316 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.652670 31321 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.657302 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa5f71c212664b78a4deb373bf5ed2f5" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:39.657984 24014 raft_consensus.cc:3060] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:16:39.664713 31321 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:39.666121 24014 raft_consensus.cc:2468] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 15.
W20260709 12:16:39.666414 23436 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.673738 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd134c48fc14e60af68e7bff27733c0" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:39.674849 24014 raft_consensus.cc:3060] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:16:39.673026 23436 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.677131 31316 raft_consensus.cc:2804] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:16:39.677779 31316 raft_consensus.cc:697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
W20260709 12:16:39.677978 23436 leader_election.cc:336] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.678778 31316 consensus_queue.cc:237] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.694360 24014 raft_consensus.cc:2468] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 19.
I20260709 12:16:39.695734 23436 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.696980 31240 raft_consensus.cc:2804] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:16:39.697414 31240 raft_consensus.cc:697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:39.698227 31240 consensus_queue.cc:237] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.727913 23214 catalog_manager.cc:5697] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 14 to 15, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 15 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:39.742975 31240 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.743690 31240 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:39.747161 31240 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:39.752192 23436 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:39.784515 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.785035 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.785835 31240 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:39.811124 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.828823 23434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:46055, user_credentials={real_user=slave}} blocked reactor thread for 44334.7us
W20260709 12:16:39.829530 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.833973 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:39.834941 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.835180 31240 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:39.835412 23434 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.837716 31316 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:39.838878 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.839342 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.839426 31298 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.840291 31298 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:39.842814 31240 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:39.844540 31298 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:39.846765 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.851220 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.852350 31240 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:39.854564 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:39.860363 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:39.861394 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.862232 31313 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:39.871901 23434 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.873848 31316 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:39.874356 31316 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:39.876531 31316 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:39.879714 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.883936 23436 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:39.890547 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:39.892426 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:39.892767 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:39.898247 31316 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:39.954537 23434 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.954547 23214 catalog_manager.cc:5697] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 18 to 19, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 19 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:39.961843 23982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111) [suppressed 507 similar messages]
I20260709 12:16:40.046891 31240 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.047456 31240 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:40.055454 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:40.059448 23436 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.059890 23436 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.061820 31316 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:40.063534 31240 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:40.080400 31298 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.080958 31298 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.083053 31298 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.087563 23980 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:40.090183 31298 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.090778 31298 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:40.104592 23980 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:40.105419 31298 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.140020 23982 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.140636 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.141496 31298 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:40.164644 31298 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.165237 31298 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.167433 31298 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.178607 23436 consensus_peers.cc:597] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:40.180747 23980 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:40.186772 23436 consensus_peers.cc:597] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:40.187597 23982 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.189762 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.190572 31298 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:40.207758 24014 raft_consensus.cc:1275] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:16:40.224668 31240 consensus_queue.cc:1048] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:16:40.230744 31316 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.231314 31316 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:40.237407 31316 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.251132 23436 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:40.252537 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:40.254302 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.255746 31316 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:40.274467 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.325214 31298 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.325949 31298 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.328703 31298 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:40.350838 24009 raft_consensus.cc:1275] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 12:16:40.352946 23980 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:40.363353 23982 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.365844 23982 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.368525 31316 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.386322 31298 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:40.382400 31316 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:40.403378 31316 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.406852 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:40.412415 23436 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:40.418367 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:40.418524 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.419775 31316 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:40.427062 31240 consensus_queue.cc:1048] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:16:40.431277 23250 debug-util.cc:398] Leaking SignalData structure 0x7b0800393860 after lost signal to thread 22110
W20260709 12:16:40.432446 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080035b6a0 after lost signal to thread 23253
W20260709 12:16:40.433857 23250 debug-util.cc:398] Leaking SignalData structure 0x7b080035d3c0 after lost signal to thread 23494
W20260709 12:16:40.435185 23250 debug-util.cc:398] Leaking SignalData structure 0x7b08003da2e0 after lost signal to thread 24040
I20260709 12:16:40.462284 31298 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.463017 31298 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.466176 31298 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:40.463514 31316 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.500234 31316 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:40.505750 31316 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:40.509318 23436 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.516587 23434 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:40.523509 23463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
W20260709 12:16:40.525629 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.540428 23436 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.545939 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:40.548601 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.549373 31332 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:40.543526 23982 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.550328 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:40.551146 22107 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 23417)
I20260709 12:16:40.551317 31298 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:40.551612 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.552210 23982 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.552929 23982 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.549014 23436 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.572319 23982 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.573084 23982 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:40.573858 31313 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:40.579535 31240 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.587087 31240 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.589012 31240 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:40.592247 23436 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.574100 23982 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.594842 23982 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.599493 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.595616 23436 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.600349 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
W20260709 12:16:40.603951 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.610082 24013 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 17.
W20260709 12:16:40.616415 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:40.619665 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.621423 31316 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:16:40.621833 31316 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.622167 31316 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:40.638576 31332 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.639197 31332 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.641283 31332 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:40.641270 31316 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.643605 31316 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:40.653582 31313 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.654189 31313 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.655491 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:40.656148 24013 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 24.
I20260709 12:16:40.656272 31313 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:40.657193 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2984a5ac8b6b49c9aa8eb969a484d610" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:40.657837 24009 raft_consensus.cc:3060] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:16:40.659034 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64" is_pre_election: true
I20260709 12:16:40.659619 23462 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54640ab8abeb4c949426d66c7e5aed87 in term 24.
W20260709 12:16:40.664822 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.666030 24009 raft_consensus.cc:2468] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 18.
W20260709 12:16:40.670274 23436 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:40.677892 23436 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.679549 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.681082 31332 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:40.671706 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 
I20260709 12:16:40.681479 31332 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.681949 31332 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:40.682192 31298 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:16:40.682651 31298 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:40.683018 31298 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:16:40.687379 23436 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:40.688252 31316 raft_consensus.cc:2804] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:16:40.688702 31316 raft_consensus.cc:697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:40.689574 31316 consensus_queue.cc:237] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.693385 31332 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:40.702620 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.702972 31332 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:40.694428 31298 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:40.705188 31298 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:40.705713 23982 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.717818 23462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "54640ab8abeb4c949426d66c7e5aed87" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "da1e2fd157894634bf92d6c82c06ec64"
I20260709 12:16:40.718603 23462 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 54640ab8abeb4c949426d66c7e5aed87 in current term 25: Already voted for candidate da1e2fd157894634bf92d6c82c06ec64 in this term.
W20260709 12:16:40.720794 23982 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:40.724050 24009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
W20260709 12:16:40.724329 23982 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.724830 24009 raft_consensus.cc:2393] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate da1e2fd157894634bf92d6c82c06ec64 in current term 25: Already voted for candidate 54640ab8abeb4c949426d66c7e5aed87 in this term.
W20260709 12:16:40.721658 23436 consensus_peers.cc:597] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.733997 23436 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:40.724836 23982 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:40.743122 31313 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:16:40.745754 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 54640ab8abeb4c949426d66c7e5aed87, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:40.751149 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:40.766680 31332 raft_consensus.cc:2749] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:16:40.777019 23982 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:40.779963 23214 catalog_manager.cc:5697] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 17 to 18, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 18 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:40.781447 23436 consensus_peers.cc:597] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.765789 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.795945 23982 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.800413 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.833734 23436 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.836392 23436 consensus_peers.cc:597] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.848547 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.849495 23982 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.862360 23436 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.862515 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.879760 23982 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.940251 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.954775 23436 consensus_peers.cc:597] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.969925 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.986812 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:40.987720 23436 consensus_peers.cc:597] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:40.997823 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:41.006084 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:41.045348 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:41.060051 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:41.139830 23436 consensus_peers.cc:597] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:41.207932 24014 raft_consensus.cc:1275] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:16:41.213205 31341 consensus_queue.cc:1048] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:16:41.217133 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:41.318907 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:41.340745 31341 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.341379 31341 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:41.345786 31341 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.349609 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:41.350167 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:41.384622 23436 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:41.385147 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.417376 31240 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:41.483843 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:41.551594 22107 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 23417)
I20260709 12:16:41.562937 31341 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.563618 31341 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:41.574229 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:41.574714 31341 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.575068 23436 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:41.575634 23436 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.576409 31341 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:41.589498 31341 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.590013 31341 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:41.592237 31341 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.600518 23436 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:41.601361 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:41.601876 23434 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.602660 31341 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:41.691896 31298 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.692500 31298 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:41.694720 31298 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.700104 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:41.701189 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:41.701603 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.704470 31298 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:41.706329 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:41.712266 31341 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.712812 31341 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:41.715024 31341 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.724202 23436 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:41.725148 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:41.725611 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.728546 31332 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:41.764626 23434 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:41.789970 31298 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.790563 31298 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:41.792567 31298 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.800239 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:41.800755 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:41.801216 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.801996 31298 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:41.837647 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:41.906059 31341 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.906625 31341 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:41.908896 31341 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.918359 23436 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:41.921170 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:41.921751 23434 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.922056 31313 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:41.922696 31313 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:41.923727 31341 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:41.934211 23980 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:41.939961 31313 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:41.962515 23982 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:41.962978 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:41.965195 31313 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:42.000855 31341 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.001468 31341 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:42.003700 31341 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.016700 23436 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:42.020412 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:42.020869 23434 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.024055 31332 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:42.077890 31332 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.078466 31332 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:42.081326 31341 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.081878 31341 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:42.084106 31341 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.096591 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.097307 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.100982 23436 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.104194 23436 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.108567 31341 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:42.100247 31332 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.119234 23436 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.119753 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.124855 31332 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:16:42.146281 31332 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.146858 31332 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:42.149128 31332 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.161703 23436 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:42.164359 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:42.164790 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.165563 31332 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:42.202332 31298 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.202840 31298 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:42.204880 31298 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.212154 23980 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.226469 23982 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.228431 23982 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.230531 31313 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:42.238308 23434 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.316205 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.367177 23436 consensus_peers.cc:597] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:42.382053 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:42.437031 31298 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.437628 31298 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:42.439688 31298 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.447937 23980 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.451237 23982 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.451735 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.453199 31313 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:42.468448 31313 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.468989 31313 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:42.471225 31313 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.478893 23980 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.492313 23434 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:42.493053 23982 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.493506 23982 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.494246 31313 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:42.552141 22107 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 23417)
W20260709 12:16:42.573028 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:42.591120 31341 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:42.593413 31341 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:42.596957 31341 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:42.601408 23434 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:42.601787 23436 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:42.613705 23436 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.614308 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.615221 31341 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:42.675464 23436 consensus_peers.cc:597] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:42.751647 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:42.763631 31341 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:42.764364 31341 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:42.772130 31341 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
W20260709 12:16:42.780668 23436 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.796569 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87" is_pre_election: true
I20260709 12:16:42.797331 24010 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 25.
I20260709 12:16:42.813839 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.815150 31341 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:16:42.815631 31341 raft_consensus.cc:493] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:42.816054 31341 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:16:42.842199 23982 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:42.850032 31341 raft_consensus.cc:515] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:42.858325 23436 leader_election.cc:336] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:42.860486 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "487ee45f93ab48478b7fa939ccf7f658" candidate_uuid: "da1e2fd157894634bf92d6c82c06ec64" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "54640ab8abeb4c949426d66c7e5aed87"
I20260709 12:16:42.861075 24010 raft_consensus.cc:3060] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:16:42.876605 24010 raft_consensus.cc:2468] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da1e2fd157894634bf92d6c82c06ec64 in term 26.
W20260709 12:16:42.877213 23436 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:42.878399 31341 leader_election.cc:290] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 election: Requested vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), 54640ab8abeb4c949426d66c7e5aed87 (127.21.150.196:40405)
I20260709 12:16:42.895733 23436 leader_election.cc:304] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87, da1e2fd157894634bf92d6c82c06ec64; no voters: 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:42.897575 31332 raft_consensus.cc:2804] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:16:42.932370 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:42.978699 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:42.994163 23982 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:43.009210 31332 raft_consensus.cc:697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 26 LEADER]: Becoming Leader. State: Replica: da1e2fd157894634bf92d6c82c06ec64, State: Running, Role: LEADER
I20260709 12:16:43.010490 31332 consensus_queue.cc:237] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:43.022312 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.035014 23436 consensus_peers.cc:597] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:43.063086 23214 catalog_manager.cc:5697] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 reported cstate change: term changed from 24 to 26, leader changed from 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194) to da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195). New cstate: current_term: 26 leader_uuid: "da1e2fd157894634bf92d6c82c06ec64" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:43.078684 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.093962 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.110790 23982 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.126003 23436 consensus_peers.cc:597] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.138738 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:43.145817 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.152127 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.152885 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.188071 23982 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.199268 23982 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.227188 23982 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.233599 23436 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.241398 23436 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.248246 23982 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.297616 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.332758 23982 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.362515 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.380836 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:43.423517 31348 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.424127 31348 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.426293 31348 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.431993 23436 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:43.445065 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.459692 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:43.460816 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:43.461990 23436 consensus_peers.cc:597] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:43.462852 23436 consensus_peers.cc:597] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:43.463369 31345 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:43.464828 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:43.477931 24010 raft_consensus.cc:1275] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 26 FOLLOWER]: Refusing update from remote peer da1e2fd157894634bf92d6c82c06ec64: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:16:43.483678 31348 consensus_queue.cc:1048] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:16:43.532632 23982 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.539306 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.552834 22107 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 23417)
W20260709 12:16:43.565255 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:43.671038 31345 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.671643 31345 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.674714 31345 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.678174 23436 consensus_peers.cc:597] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.680197 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.683313 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:43.682785 23436 consensus_peers.cc:597] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.700588 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:43.709472 31348 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.710085 31348 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.712162 31348 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.713140 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:43.727734 23436 consensus_peers.cc:597] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:43.732708 23436 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.734923 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:43.737149 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:43.740301 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:43.741868 23436 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:43.742318 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:43.743049 31332 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:43.748831 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:43.743283 23436 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:43.749821 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:43.750578 31332 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:16:43.777251 31298 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.777797 31298 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:43.779636 31298 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.784510 23980 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:43.817616 23982 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:43.818154 23982 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:43.818888 31298 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:16:43.843176 31345 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.843827 31345 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.846182 31345 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.863466 23436 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:43.865314 23434 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:43.865824 23434 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:43.866768 31345 raft_consensus.cc:2749] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:43.907300 31348 raft_consensus.cc:493] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.907894 31348 raft_consensus.cc:515] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.910157 31348 leader_election.cc:290] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:43.914237 31348 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:43.916580 31348 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:43.918951 31348 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:43.915979 23436 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:43.934628 23436 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:43.936916 23434 leader_election.cc:336] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:43.941375 23434 leader_election.cc:304] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:43.942907 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:43.943849 23434 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:43.942045 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:43.943393 31348 raft_consensus.cc:2749] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:43.959060 31345 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:44.065372 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:44.105882 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:44.124464 31345 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:44.125211 31345 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:44.127377 31345 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:44.129957 31345 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:44.130532 31345 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:44.145974 31345 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:44.158535 23434 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:44.171015 23436 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:44.177204 23436 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:44.185571 31345 raft_consensus.cc:2749] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:16:44.188566 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:44.260500 23436 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:44.262976 23436 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:44.273983 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:44.274648 31345 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:16:44.359403 31360 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:44.360054 31360 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:44.362244 31360 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:44.367712 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:44.416718 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:44.417514 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:44.418581 31360 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:44.423784 31360 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:44.424458 31360 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:44.427160 31360 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:44.434206 23980 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:44.441327 23982 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:44.442034 23982 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:44.443024 31360 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:44.482460 23436 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40405, user_credentials={real_user=slave}} blocked reactor thread for 112963us
W20260709 12:16:44.553548 22107 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 23417)
W20260709 12:16:44.630957 23434 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:44.642390 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:44.704473 31348 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:44.706646 31348 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:44.723624 23434 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:44.725000 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:44.729365 31348 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:44.746433 23436 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:44.747084 23436 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:44.747993 31348 raft_consensus.cc:2749] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:45.002785 31298 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.003509 31298 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:45.006717 31298 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.008234 23980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111) [suppressed 609 similar messages]
W20260709 12:16:45.012369 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:45.036904 31359 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.037598 31359 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:45.039791 31359 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.048287 23980 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:45.138016 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:45.139611 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.140439 31298 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:45.146620 23982 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:45.147284 23982 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.151572 31360 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:45.182029 23434 consensus_peers.cc:597] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.183699 23434 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.190148 23434 consensus_peers.cc:597] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.202631 23436 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.236384 23436 consensus_peers.cc:597] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:45.262897 23982 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.271014 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.308997 23436 consensus_peers.cc:597] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:45.365797 23436 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.449915 23982 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.485193 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:45.489080 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800366d60 after lost signal to thread 22110
W20260709 12:16:45.490301 23491 debug-util.cc:398] Leaking SignalData structure 0x7b0800399940 after lost signal to thread 23253
W20260709 12:16:45.491621 23491 debug-util.cc:398] Leaking SignalData structure 0x7b080038bc20 after lost signal to thread 23494
W20260709 12:16:45.492592 23491 debug-util.cc:398] Leaking SignalData structure 0x7b08003c5340 after lost signal to thread 24040
I20260709 12:16:45.496628 31359 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.497351 31359 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:45.499830 31359 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.505203 23982 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:45.507905 23980 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:45.508950 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.509903 31359 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:45.520692 31367 raft_consensus.cc:493] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.521363 31367 raft_consensus.cc:515] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:45.527205 31367 leader_election.cc:290] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.528302 23434 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:45.531776 23436 leader_election.cc:336] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:45.532261 23436 leader_election.cc:304] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.533607 31367 raft_consensus.cc:2749] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:45.554117 22107 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 23417)
W20260709 12:16:45.562754 23982 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.580885 23982 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.605935 23436 consensus_peers.cc:597] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.637111 23982 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:45.667676 31367 raft_consensus.cc:493] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.670617 31367 raft_consensus.cc:515] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:45.672768 31367 leader_election.cc:290] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.691699 23434 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:45.700723 23436 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.704728 23436 leader_election.cc:336] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:45.706212 23436 leader_election.cc:304] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.708338 31361 raft_consensus.cc:2749] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:16:45.709826 23436 consensus_peers.cc:597] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.710053 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.734165 23982 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.739425 23436 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.766222 23980 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.772150 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.805554 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.837057 23434 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:45.873466 23982 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.884974 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:45.919646 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:16:45.930267 31348 raft_consensus.cc:493] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:45.930896 31348 raft_consensus.cc:515] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
W20260709 12:16:45.932176 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:45.933034 31348 leader_election.cc:290] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:45.938162 23436 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:45.939793 23434 leader_election.cc:336] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:45.940267 23434 leader_election.cc:304] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:45.942906 31348 raft_consensus.cc:2749] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:16:45.944859 23436 consensus_peers.cc:597] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.025282 23982 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.027302 23980 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.039330 23436 consensus_peers.cc:597] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.059036 23434 consensus_peers.cc:597] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.077040 24037 debug-util.cc:398] Leaking SignalData structure 0x7b080037cba0 after lost signal to thread 22110
W20260709 12:16:46.078203 24037 debug-util.cc:398] Leaking SignalData structure 0x7b08002a39e0 after lost signal to thread 23253
W20260709 12:16:46.078799 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.079506 24037 debug-util.cc:398] Leaking SignalData structure 0x7b080024a200 after lost signal to thread 23494
W20260709 12:16:46.081468 24037 debug-util.cc:398] Leaking SignalData structure 0x7b0800375e20 after lost signal to thread 24040
W20260709 12:16:46.145794 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.159104 23436 consensus_peers.cc:597] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.165740 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.174326 23436 consensus_peers.cc:597] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:46.187907 23434 consensus_peers.cc:597] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:46.205580 23436 consensus_peers.cc:597] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:46.234845 31371 raft_consensus.cc:493] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:46.235402 31371 raft_consensus.cc:515] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:46.240432 31371 leader_election.cc:290] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:46.244740 23434 consensus_peers.cc:597] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.245378 23434 consensus_peers.cc:597] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.245890 23434 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:46.254616 23980 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:46.255089 23980 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.257709 23436 leader_election.cc:336] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:46.258286 23436 leader_election.cc:304] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:46.262360 31371 raft_consensus.cc:2749] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:16:46.264442 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.295328 23980 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:46.323801 23436 consensus_peers.cc:597] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.325090 23434 consensus_peers.cc:597] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.339284 23434 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.354053 23434 consensus_peers.cc:597] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:16:46.427414 23982 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:16:46.474614 23434 consensus_peers.cc:597] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:46.555120 22107 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 23417)
W20260709 12:16:46.557145 23434 consensus_peers.cc:597] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:46.654614 31359 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:46.655722 31359 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:46.666985 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:46.669883 31359 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:46.709033 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:46.709534 23982 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:46.711329 31369 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:46.721596 23434 consensus_peers.cc:597] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:16:46.768975 31370 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:46.769908 31370 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:46.772418 31370 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:46.803884 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:46.832173 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:46.832710 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:46.833599 31370 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:16:47.059907 31367 raft_consensus.cc:493] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.060593 31367 raft_consensus.cc:515] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:47.062796 31367 leader_election.cc:290] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.069376 23434 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:47.071239 23436 leader_election.cc:336] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:47.071795 23436 leader_election.cc:304] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:47.073784 31367 raft_consensus.cc:2749] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:16:47.112558 23434 consensus_peers.cc:597] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:47.165930 31367 raft_consensus.cc:493] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.166643 31367 raft_consensus.cc:515] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:47.168879 31367 leader_election.cc:290] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.176911 23436 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:47.182142 23434 leader_election.cc:336] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:47.182593 23434 leader_election.cc:304] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:47.184739 31361 raft_consensus.cc:2749] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:47.268870 23434 consensus_peers.cc:597] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:16:47.444438 31367 raft_consensus.cc:493] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.445056 31367 raft_consensus.cc:515] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:47.446964 31367 leader_election.cc:290] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.450434 23436 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:47.526734 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
I20260709 12:16:47.527771 31373 raft_consensus.cc:493] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.528577 31373 raft_consensus.cc:515] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:47.531157 31373 leader_election.cc:290] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.537086 23982 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:47.566490 31361 raft_consensus.cc:493] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.567155 31361 raft_consensus.cc:515] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:47.569504 31361 leader_election.cc:290] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.573239 23436 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:47.654596 31373 raft_consensus.cc:493] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.655284 31373 raft_consensus.cc:515] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:47.657712 31373 leader_election.cc:290] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.663301 23982 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:47.665073 23436 consensus_peers.cc:597] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:47.670228 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:16:47.707216 31361 raft_consensus.cc:493] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:47.707842 31361 raft_consensus.cc:515] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } }
I20260709 12:16:47.709724 31361 leader_election.cc:290] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:47.716636 23436 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:47.718585 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:47.719660 22107 tablet_replica.cc:333] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.720679 22107 raft_consensus.cc:2243] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.721796 22107 raft_consensus.cc:2272] T d2bbbae982b14adea2da3b5b5217b18e P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.725678 22107 tablet_replica.cc:333] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.726650 22107 raft_consensus.cc:2243] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.727310 22107 raft_consensus.cc:2272] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.730147 22107 tablet_replica.cc:333] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.730940 22107 raft_consensus.cc:2243] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.732178 22107 raft_consensus.cc:2272] T 350816ae62fd4fb598a9ea6c84adf933 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.735363 22107 tablet_replica.cc:333] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.736244 22107 raft_consensus.cc:2243] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.737136 22107 raft_consensus.cc:2272] T c5d8af85525845a192206b43a61f5c41 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.740480 22107 tablet_replica.cc:333] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.741465 22107 raft_consensus.cc:2243] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 11 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.742511 22107 raft_consensus.cc:2272] T 9689b2f60f934eb6b1c067507423b816 P da1e2fd157894634bf92d6c82c06ec64 [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.745863 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:47.746567 23982 consensus_peers.cc:597] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:47.746886 22107 tablet_replica.cc:333] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.747820 22107 raft_consensus.cc:2243] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.748471 22107 raft_consensus.cc:2272] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.751101 22107 tablet_replica.cc:333] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.751709 22107 raft_consensus.cc:2243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.753113 22107 raft_consensus.cc:2272] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.755498 22107 tablet_replica.cc:333] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.756117 22107 raft_consensus.cc:2243] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.757818 22107 raft_consensus.cc:2272] T fa5f71c212664b78a4deb373bf5ed2f5 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.760623 22107 tablet_replica.cc:333] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.761394 22107 raft_consensus.cc:2243] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.762557 22107 raft_consensus.cc:2272] T e3dffd4f7c72413b84a3862372dcce85 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.763934 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.766095 22107 tablet_replica.cc:333] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.766940 22107 raft_consensus.cc:2243] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.767933 22107 raft_consensus.cc:2272] T 1376b20c4a2b47efbb5cd4afdb7f2ea4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.770157 23436 consensus_peers.cc:597] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:47.771596 22107 tablet_replica.cc:333] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.772495 22107 raft_consensus.cc:2243] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.773120 22107 raft_consensus.cc:2272] T 271ce3c0dde64506a60105c55526e4a3 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.774183 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.775313 22107 tablet_replica.cc:333] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.775965 22107 raft_consensus.cc:2243] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.777334 22107 raft_consensus.cc:2272] T abd134c48fc14e60af68e7bff27733c0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.779983 22107 tablet_replica.cc:333] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
W20260709 12:16:47.780608 23982 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.780725 22107 raft_consensus.cc:2243] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.781726 22107 raft_consensus.cc:2272] T 5e2145c1046c436e8ca4a54ef0df4ab5 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.784224 22107 tablet_replica.cc:333] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.784878 22107 raft_consensus.cc:2243] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.786002 22107 raft_consensus.cc:2272] T 19d9facb5378417aaa3279b3dd9f02a9 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.788642 22107 tablet_replica.cc:333] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.789376 22107 raft_consensus.cc:2243] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.789974 22107 raft_consensus.cc:2272] T f55bd3b311de43ba8aca6be22937f68a P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.791922 22107 tablet_replica.cc:333] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.792459 22107 raft_consensus.cc:2243] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.792924 22107 raft_consensus.cc:2272] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.795360 22107 tablet_replica.cc:333] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.796041 22107 raft_consensus.cc:2243] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.797003 22107 raft_consensus.cc:2272] T b055a413d01c460895f3bc5034afc461 P da1e2fd157894634bf92d6c82c06ec64 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.797731 23982 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.799008 22107 tablet_replica.cc:333] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.799609 22107 raft_consensus.cc:2243] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.800103 22107 raft_consensus.cc:2272] T de7238eb389343cd954cb756bed5925f P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.802084 22107 tablet_replica.cc:333] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.802613 22107 raft_consensus.cc:2243] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.803109 22107 raft_consensus.cc:2272] T dcbe5680eae64fa2a2565e9b0973703d P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.804862 22107 tablet_replica.cc:333] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.805398 22107 raft_consensus.cc:2243] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.805953 22107 raft_consensus.cc:2272] T 29bc5916fff44ea3b1dcd31091d822b9 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.808490 22107 tablet_replica.cc:333] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.809057 22107 raft_consensus.cc:2243] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.809638 22107 raft_consensus.cc:2272] T a658e02588314c9ca8a73f5eb423fb27 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.811503 22107 tablet_replica.cc:333] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.812057 22107 raft_consensus.cc:2243] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.812796 22107 raft_consensus.cc:2272] T 1399c0b792a048cda2bcae40b23432c0 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.814635 22107 tablet_replica.cc:333] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.815274 22107 raft_consensus.cc:2243] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.816865 22107 raft_consensus.cc:2272] T cfaa0d3f687341a1ac860baa49e8bfb4 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.819226 22107 tablet_replica.cc:333] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.819751 22107 raft_consensus.cc:2243] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.820786 22107 raft_consensus.cc:2272] T ba09297a7e5c4b4f8b873c36c70e2a2d P da1e2fd157894634bf92d6c82c06ec64 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.823493 22107 tablet_replica.cc:333] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.824229 22107 raft_consensus.cc:2243] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.825585 22107 raft_consensus.cc:2272] T 32256a541626459187bbe0f5f7e6643c P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.827535 22107 tablet_replica.cc:333] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.828063 22107 raft_consensus.cc:2243] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.828685 22107 raft_consensus.cc:2272] T 88fa82a6fcc0481fa12dd9a6f1782ab3 P da1e2fd157894634bf92d6c82c06ec64 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.831061 22107 tablet_replica.cc:333] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.831705 22107 raft_consensus.cc:2243] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.832831 22107 raft_consensus.cc:2272] T 487ee45f93ab48478b7fa939ccf7f658 P da1e2fd157894634bf92d6c82c06ec64 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.833405 23982 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:47.835104 22107 tablet_replica.cc:333] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.835767 22107 raft_consensus.cc:2243] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.836645 22107 raft_consensus.cc:2272] T 2349832eeb6f4824800ff39a557daeb6 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.838624 22107 tablet_replica.cc:333] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.839148 22107 raft_consensus.cc:2243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.839809 22107 raft_consensus.cc:2272] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P da1e2fd157894634bf92d6c82c06ec64 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.841676 22107 tablet_replica.cc:333] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.842299 22107 raft_consensus.cc:2243] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.842970 22107 raft_consensus.cc:2272] T 03644bf5f16347c98fd9217f8a7fab79 P da1e2fd157894634bf92d6c82c06ec64 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.846131 22107 tablet_replica.cc:333] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.846899 22107 raft_consensus.cc:2243] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.847805 22107 raft_consensus.cc:2272] T 746c120ff6e048eaa7b3a03be2b8a91e P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.850165 22107 tablet_replica.cc:333] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.850688 22107 raft_consensus.cc:2243] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.851609 22107 raft_consensus.cc:2272] T 57b150e0c7494c95beb04642f466f450 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.853998 22107 tablet_replica.cc:333] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.854693 22107 raft_consensus.cc:2243] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.855561 22107 raft_consensus.cc:2272] T 6d83783f67774af1b007547d0aca8da1 P da1e2fd157894634bf92d6c82c06ec64 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.857645 22107 tablet_replica.cc:333] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.858105 22107 raft_consensus.cc:2243] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.858767 22107 raft_consensus.cc:2272] T 06d137914eb14302bb888a191819fb75 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.860518 22107 tablet_replica.cc:333] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.860999 22107 raft_consensus.cc:2243] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.861881 22107 raft_consensus.cc:2272] T 19b18a93b6e8478e9f44ce88b342ab54 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.863704 22107 tablet_replica.cc:333] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.864169 22107 raft_consensus.cc:2243] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 18 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.865406 22107 raft_consensus.cc:2272] T 2984a5ac8b6b49c9aa8eb969a484d610 P da1e2fd157894634bf92d6c82c06ec64 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.867246 22107 tablet_replica.cc:333] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.867682 22107 raft_consensus.cc:2243] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 13 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.868834 22107 raft_consensus.cc:2272] T c192a749705545f298d61bbb8ccb5665 P da1e2fd157894634bf92d6c82c06ec64 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.869660 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:16:47.870968 22107 tablet_replica.cc:333] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.871559 22107 raft_consensus.cc:2243] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.872123 22107 raft_consensus.cc:2272] T ff06250da5d340babc2a28fd336380a7 P da1e2fd157894634bf92d6c82c06ec64 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.874011 22107 tablet_replica.cc:333] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.874537 22107 raft_consensus.cc:2243] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.875245 22107 raft_consensus.cc:2272] T d736b5901fbd4de09b83cc09a8d00fd1 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.877120 22107 tablet_replica.cc:333] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.877712 22107 raft_consensus.cc:2243] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.878163 22107 raft_consensus.cc:2272] T c4de7d7863704646b55025720cc0da9c P da1e2fd157894634bf92d6c82c06ec64 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.879909 22107 tablet_replica.cc:333] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.880415 22107 raft_consensus.cc:2243] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.881098 22107 raft_consensus.cc:2272] T ea456647290e4959b61a6fa23ecfbfd4 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.882952 22107 tablet_replica.cc:333] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.883440 22107 raft_consensus.cc:2243] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.884003 22107 raft_consensus.cc:2272] T c3d1af6a7ce14f5387824508ce04bfd0 P da1e2fd157894634bf92d6c82c06ec64 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.885829 22107 tablet_replica.cc:333] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.886322 22107 raft_consensus.cc:2243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.886799 22107 raft_consensus.cc:2272] T b4ebaab4fff4457a89f5a00d4f71ddc8 P da1e2fd157894634bf92d6c82c06ec64 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.888628 22107 tablet_replica.cc:333] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64: stopping tablet replica
I20260709 12:16:47.889089 22107 raft_consensus.cc:2243] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.889674 22107 raft_consensus.cc:2272] T a59619d146f04895b0e204f6e4bdcf47 P da1e2fd157894634bf92d6c82c06ec64 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:47.898465 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:47.900167 23982 consensus_peers.cc:597] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:47.904269 23982 consensus_peers.cc:597] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:47.924548 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:47.974334 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.008697 23982 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.013844 23982 consensus_peers.cc:597] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.048790 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.049750 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.085355 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.093607 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.107450 23982 consensus_peers.cc:597] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.157042 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.160578 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.166680 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:48.169868 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.175704 23982 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Couldn't send request to peer da1e2fd157894634bf92d6c82c06ec64. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:48.190718 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.253787 23433 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.264566 23434 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.270910 23436 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.280002 23982 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.287436 23435 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:16:48.307896 31207 leader_election.cc:336] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:48.308434 31207 leader_election.cc:304] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:48.309121 31207 raft_consensus.cc:2686] T 2abf7c5535ea498fa1d3c949bcdc25ca P da1e2fd157894634bf92d6c82c06ec64: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:16:48.313031 23980 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:48.313766 23980 leader_election.cc:336] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:48.314216 23980 leader_election.cc:304] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:48.315007 31373 raft_consensus.cc:2749] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:48.315191 23980 leader_election.cc:336] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:48.315662 23980 leader_election.cc:304] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:48.316215 31207 leader_election.cc:336] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:48.316308 31373 raft_consensus.cc:2749] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:16:48.316577 23980 consensus_peers.cc:597] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:48.316722 31207 leader_election.cc:304] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:48.317399 31207 raft_consensus.cc:2686] T 2cd3243d8c2d447ca873f0890f2c0058 P da1e2fd157894634bf92d6c82c06ec64: unable to run election callback: Service unavailable: The pool has been shut down.
I20260709 12:16:48.319720 31373 raft_consensus.cc:493] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
W20260709 12:16:48.320271 31207 leader_election.cc:336] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Service unavailable: reactor is shutting down (error 108)
I20260709 12:16:48.320243 31373 raft_consensus.cc:515] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.320672 31207 leader_election.cc:304] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da1e2fd157894634bf92d6c82c06ec64; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
W20260709 12:16:48.321172 31207 raft_consensus.cc:2686] T 6f8cb992d1f8470a8b883e9287064025 P da1e2fd157894634bf92d6c82c06ec64: unable to run election callback: Service unavailable: The pool has been shut down.
I20260709 12:16:48.322156 31373 leader_election.cc:290] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:48.325462 23980 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:48.327158 23982 leader_election.cc:336] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:48.327508 23982 leader_election.cc:304] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:48.328086 31373 raft_consensus.cc:2749] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:48.338609 31373 raft_consensus.cc:493] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:48.339118 31373 raft_consensus.cc:515] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.340966 31373 leader_election.cc:290] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:48.352210 23982 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:48.356451 23980 leader_election.cc:336] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:48.356870 23980 leader_election.cc:304] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:48.357971 31373 raft_consensus.cc:2749] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:16:48.365182 23982 consensus_peers.cc:597] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.384312 23980 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:16:48.439983 23982 consensus_peers.cc:597] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.484427 23982 consensus_peers.cc:597] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.522020 23982 consensus_peers.cc:597] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.539909 23982 consensus_peers.cc:597] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:48.623737 31373 raft_consensus.cc:493] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:48.624269 31373 raft_consensus.cc:515] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.626297 31373 leader_election.cc:290] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
W20260709 12:16:48.630481 23980 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:48.631318 23982 leader_election.cc:336] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:48.631652 23982 leader_election.cc:304] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:48.632252 31373 raft_consensus.cc:2749] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:16:48.749294 23982 consensus_peers.cc:597] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:16:48.823555 31373 raft_consensus.cc:493] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.824115 31373 raft_consensus.cc:515] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.826273 31373 leader_election.cc:290] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.830003 23982 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
W20260709 12:16:48.830967 23980 leader_election.cc:336] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:48.831295 23980 leader_election.cc:304] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.831868 31373 raft_consensus.cc:2749] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:48.832898 31373 raft_consensus.cc:493] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:16:48.833549 23980 consensus_peers.cc:597] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:48.833367 31373 raft_consensus.cc:515] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.835310 31373 leader_election.cc:290] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.839238 23980 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:48.839679 23982 leader_election.cc:336] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:48.840076 23982 leader_election.cc:304] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.840698 31373 raft_consensus.cc:2749] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:16:48.848940 31373 raft_consensus.cc:493] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.849455 31373 raft_consensus.cc:515] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.851681 31373 leader_election.cc:290] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.855592 23982 leader_election.cc:336] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
W20260709 12:16:48.856249 23982 leader_election.cc:336] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:48.856578 23982 leader_election.cc:304] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.857160 31373 raft_consensus.cc:2749] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:48.862011 31373 raft_consensus.cc:493] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.862427 31373 raft_consensus.cc:515] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.864271 31373 leader_election.cc:290] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.867909 23982 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:48.868626 23982 leader_election.cc:336] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:48.869001 23982 leader_election.cc:304] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.869694 31373 raft_consensus.cc:2749] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:16:48.890861 23980 consensus_peers.cc:597] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:48.909233 31370 raft_consensus.cc:493] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.909853 31370 raft_consensus.cc:515] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.911923 31370 leader_election.cc:290] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.914623 23982 consensus_peers.cc:597] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Couldn't send request to peer 6253b834ffb744579b46fcede2fc0e91. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:16:48.916224 23980 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:48.916937 23982 leader_election.cc:336] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:48.917274 23982 leader_election.cc:304] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.917959 31370 raft_consensus.cc:2749] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:48.939044 31370 raft_consensus.cc:493] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.939603 31370 raft_consensus.cc:515] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.941664 31370 leader_election.cc:290] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.947913 23980 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:48.948558 23982 leader_election.cc:336] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:48.948952 23982 leader_election.cc:304] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:48.950273 23980 consensus_peers.cc:597] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:48.950372 31370 raft_consensus.cc:2749] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:48.965867 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
W20260709 12:16:48.971489 23980 consensus_peers.cc:597] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:48.976605 31370 raft_consensus.cc:493] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:48.977345 31370 raft_consensus.cc:515] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:48.979625 31370 leader_election.cc:290] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:48.983588 23982 leader_election.cc:336] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:48.984313 23982 leader_election.cc:336] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:48.984694 23982 leader_election.cc:304] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:48.985503 31370 raft_consensus.cc:2749] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:16:49.034858 23980 consensus_peers.cc:597] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 -> Peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Couldn't send request to peer 2778b449761a46708f2f643b1b5b5854. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:16:49.041025 31370 raft_consensus.cc:493] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.041743 31370 raft_consensus.cc:515] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.044373 31370 leader_election.cc:290] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:49.049269 23982 leader_election.cc:336] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
W20260709 12:16:49.049904 23980 leader_election.cc:336] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:49.050299 23980 leader_election.cc:304] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.050979 31370 raft_consensus.cc:2749] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:49.054252 31370 raft_consensus.cc:493] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.054782 31370 raft_consensus.cc:515] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.057199 31370 leader_election.cc:290] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.060186 31370 raft_consensus.cc:493] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.061476 31373 raft_consensus.cc:493] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:16:49.061589 23982 leader_election.cc:336] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:49.060755 31370 raft_consensus.cc:515] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.062139 31373 raft_consensus.cc:515] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:49.062467 23982 leader_election.cc:336] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.063097 23982 leader_election.cc:304] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.064009 31375 raft_consensus.cc:2749] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:49.064265 31370 leader_election.cc:290] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.064394 31373 leader_election.cc:290] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:49.069133 23982 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:49.069757 23982 leader_election.cc:336] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:49.070322 23982 leader_election.cc:336] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.070659 23982 leader_election.cc:304] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
W20260709 12:16:49.071336 23982 leader_election.cc:336] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.071274 31373 raft_consensus.cc:2749] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:16:49.071839 23982 leader_election.cc:304] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.072585 31373 raft_consensus.cc:2749] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:16:49.076623 31373 raft_consensus.cc:493] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.077162 31373 raft_consensus.cc:515] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.079015 31373 leader_election.cc:290] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.080212 22107 tablet_server.cc:179] TabletServer@127.21.150.196:0 shutting down...
W20260709 12:16:49.084024 23982 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
W20260709 12:16:49.085330 23980 leader_election.cc:336] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:49.085754 23980 leader_election.cc:304] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.086460 31370 raft_consensus.cc:2749] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:16:49.093281 31370 raft_consensus.cc:493] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6253b834ffb744579b46fcede2fc0e91)
I20260709 12:16:49.093852 31370 raft_consensus.cc:515] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.096256 31370 leader_election.cc:290] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721)
I20260709 12:16:49.102082 31370 raft_consensus.cc:493] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
W20260709 12:16:49.102485 23982 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
I20260709 12:16:49.102659 31370 raft_consensus.cc:515] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
W20260709 12:16:49.104365 23980 leader_election.cc:336] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:49.104820 23980 leader_election.cc:304] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, 6253b834ffb744579b46fcede2fc0e91
I20260709 12:16:49.104910 31370 leader_election.cc:290] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.107510 31370 raft_consensus.cc:2749] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:16:49.111109 23982 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
W20260709 12:16:49.112946 23980 leader_election.cc:336] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:49.116012 23980 leader_election.cc:304] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.117008 31370 raft_consensus.cc:2749] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:16:49.137872 31370 raft_consensus.cc:493] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.138485 31370 raft_consensus.cc:515] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.141137 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:49.141376 31370 leader_election.cc:290] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
W20260709 12:16:49.145591 23980 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
I20260709 12:16:49.142109 22107 tablet_replica.cc:333] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.146906 22107 raft_consensus.cc:2243] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.147962 22107 raft_consensus.cc:2272] T 97dc7e2c632a45499346d6fcf8e4aee9 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:49.148232 23982 leader_election.cc:336] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.148658 23982 leader_election.cc:304] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.149298 31370 raft_consensus.cc:2749] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:16:49.151122 22107 tablet_replica.cc:333] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.151835 22107 raft_consensus.cc:2243] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.152547 22107 raft_consensus.cc:2272] T cfaa0d3f687341a1ac860baa49e8bfb4 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.154824 22107 tablet_replica.cc:333] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.155416 22107 raft_consensus.cc:2243] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.156031 22107 raft_consensus.cc:2272] T fa5f71c212664b78a4deb373bf5ed2f5 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.158349 22107 tablet_replica.cc:333] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.158851 22107 raft_consensus.cc:2243] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.159343 22107 raft_consensus.cc:2272] T 87e270393fc841ddac0d9170b0bc6554 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.161154 22107 tablet_replica.cc:333] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.161777 22107 raft_consensus.cc:2243] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.162238 22107 raft_consensus.cc:2272] T b055a413d01c460895f3bc5034afc461 P 54640ab8abeb4c949426d66c7e5aed87 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.164250 22107 tablet_replica.cc:333] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.164774 22107 raft_consensus.cc:2243] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.165704 22107 raft_consensus.cc:2272] T 9d236b23181b45e5a16e91a3b6a30058 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.167567 22107 tablet_replica.cc:333] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.168076 22107 raft_consensus.cc:2243] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.168624 31370 raft_consensus.cc:493] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.169196 22107 raft_consensus.cc:2272] T 1bca0d75a0cb485ebd79c619c3bf8341 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.169183 31370 raft_consensus.cc:515] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6253b834ffb744579b46fcede2fc0e91" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 41721 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.171586 31370 leader_election.cc:290] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.172286 22107 tablet_replica.cc:333] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.173214 22107 raft_consensus.cc:2243] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.174351 22107 raft_consensus.cc:2272] T ec4e3b5eed804f2a8c2a01e5d1ae5f53 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:49.174501 23982 leader_election.cc:336] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 6253b834ffb744579b46fcede2fc0e91 (127.21.150.194:41721): Network error: Client connection negotiation failed: client connection to 127.21.150.194:41721: connect: Connection refused (error 111)
W20260709 12:16:49.176730 23982 leader_election.cc:336] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.177138 23982 leader_election.cc:304] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 6253b834ffb744579b46fcede2fc0e91, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.177459 22107 tablet_replica.cc:333] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.178001 31370 raft_consensus.cc:2749] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:16:49.178359 22107 raft_consensus.cc:2243] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.179585 22107 raft_consensus.cc:2272] T 709006b08c044ed38e2df81795ed1613 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.182212 22107 tablet_replica.cc:333] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.182907 22107 raft_consensus.cc:2243] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.183310 22107 raft_consensus.cc:2272] T 487ee45f93ab48478b7fa939ccf7f658 P 54640ab8abeb4c949426d66c7e5aed87 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.185144 22107 tablet_replica.cc:333] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.185708 22107 raft_consensus.cc:2243] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.186560 22107 raft_consensus.cc:2272] T baab486079784de4933d5f96a0a5321b P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.188403 22107 tablet_replica.cc:333] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.188891 22107 raft_consensus.cc:2243] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.189713 22107 raft_consensus.cc:2272] T 22fc2096e78f47158bbb4bae3a474e2c P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.191452 22107 tablet_replica.cc:333] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.191918 22107 raft_consensus.cc:2243] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.192307 22107 raft_consensus.cc:2272] T 6d83783f67774af1b007547d0aca8da1 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.194221 22107 tablet_replica.cc:333] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.194660 22107 raft_consensus.cc:2243] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.195065 22107 raft_consensus.cc:2272] T 57b150e0c7494c95beb04642f466f450 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.196732 22107 tablet_replica.cc:333] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.197211 22107 raft_consensus.cc:2243] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.197662 22107 raft_consensus.cc:2272] T d736b5901fbd4de09b83cc09a8d00fd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.199488 22107 tablet_replica.cc:333] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.199959 22107 raft_consensus.cc:2243] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.200388 22107 raft_consensus.cc:2272] T fb9a5906264b4cae9c8e888ad57c13c8 P 54640ab8abeb4c949426d66c7e5aed87 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.202273 22107 tablet_replica.cc:333] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.202735 22107 raft_consensus.cc:2243] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.203135 22107 raft_consensus.cc:2272] T ea456647290e4959b61a6fa23ecfbfd4 P 54640ab8abeb4c949426d66c7e5aed87 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.204901 22107 tablet_replica.cc:333] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.205412 22107 raft_consensus.cc:2243] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.206151 22107 raft_consensus.cc:2272] T c4de7d7863704646b55025720cc0da9c P 54640ab8abeb4c949426d66c7e5aed87 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.207988 22107 tablet_replica.cc:333] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.208467 22107 raft_consensus.cc:2243] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.209065 22107 raft_consensus.cc:2272] T ec46374214e6444fa4c5afabe5bb309a P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.210999 22107 tablet_replica.cc:333] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.211455 22107 raft_consensus.cc:2243] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.212260 22107 raft_consensus.cc:2272] T de7238eb389343cd954cb756bed5925f P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.214092 22107 tablet_replica.cc:333] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.214555 22107 raft_consensus.cc:2243] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.214938 22107 raft_consensus.cc:2272] T abd134c48fc14e60af68e7bff27733c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.216817 22107 tablet_replica.cc:333] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.217403 22107 raft_consensus.cc:2243] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.217814 22107 raft_consensus.cc:2272] T 9689b2f60f934eb6b1c067507423b816 P 54640ab8abeb4c949426d66c7e5aed87 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.219507 22107 tablet_replica.cc:333] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.220064 22107 raft_consensus.cc:2243] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.220994 22107 raft_consensus.cc:2272] T b4ebaab4fff4457a89f5a00d4f71ddc8 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.223263 22107 tablet_replica.cc:333] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.224062 22107 raft_consensus.cc:2243] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.224813 31373 raft_consensus.cc:493] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader da1e2fd157894634bf92d6c82c06ec64)
I20260709 12:16:49.225040 22107 raft_consensus.cc:2272] T 03644bf5f16347c98fd9217f8a7fab79 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.225407 31373 raft_consensus.cc:515] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "2778b449761a46708f2f643b1b5b5854" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 46055 } } peers { permanent_uuid: "da1e2fd157894634bf92d6c82c06ec64" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 33909 } } peers { permanent_uuid: "54640ab8abeb4c949426d66c7e5aed87" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40405 } attrs { promote: false } }
I20260709 12:16:49.227859 22107 tablet_replica.cc:333] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.228080 31373 leader_election.cc:290] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055), da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909)
I20260709 12:16:49.228695 22107 raft_consensus.cc:2243] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.229875 22107 raft_consensus.cc:2272] T a658e02588314c9ca8a73f5eb423fb27 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:49.230767 23980 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 2778b449761a46708f2f643b1b5b5854 (127.21.150.193:46055): Network error: Client connection negotiation failed: client connection to 127.21.150.193:46055: connect: Connection refused (error 111)
W20260709 12:16:49.231950 23982 leader_election.cc:336] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer da1e2fd157894634bf92d6c82c06ec64 (127.21.150.195:33909): Network error: Client connection negotiation failed: client connection to 127.21.150.195:33909: connect: Connection refused (error 111)
I20260709 12:16:49.232311 22107 tablet_replica.cc:333] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.232306 23982 leader_election.cc:304] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54640ab8abeb4c949426d66c7e5aed87; no voters: 2778b449761a46708f2f643b1b5b5854, da1e2fd157894634bf92d6c82c06ec64
I20260709 12:16:49.233006 31373 raft_consensus.cc:2749] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:16:49.233086 22107 raft_consensus.cc:2243] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.234174 22107 raft_consensus.cc:2272] T 7c7ea21ce0804c45ac69bf6430bcfb45 P 54640ab8abeb4c949426d66c7e5aed87 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.236513 22107 tablet_replica.cc:333] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.237296 22107 raft_consensus.cc:2243] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.237905 22107 raft_consensus.cc:2272] T 746c120ff6e048eaa7b3a03be2b8a91e P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.240006 22107 tablet_replica.cc:333] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.240563 22107 raft_consensus.cc:2243] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.241014 22107 raft_consensus.cc:2272] T 446014759a9c4a0f9c99759951367083 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.242830 22107 tablet_replica.cc:333] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.243526 22107 raft_consensus.cc:2243] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:49.244169 31373 raft_consensus.cc:670] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:16:49.244282 22107 raft_consensus.cc:2272] T 2349832eeb6f4824800ff39a557daeb6 P 54640ab8abeb4c949426d66c7e5aed87 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.246658 22107 tablet_replica.cc:333] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.247190 22107 raft_consensus.cc:2243] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.247933 22107 raft_consensus.cc:2272] T c3d1af6a7ce14f5387824508ce04bfd0 P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.250157 22107 tablet_replica.cc:333] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.250699 22107 raft_consensus.cc:2243] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.251137 22107 raft_consensus.cc:2272] T 32256a541626459187bbe0f5f7e6643c P 54640ab8abeb4c949426d66c7e5aed87 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.252979 22107 tablet_replica.cc:333] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.253563 22107 raft_consensus.cc:2243] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.253995 22107 raft_consensus.cc:2272] T 1399c0b792a048cda2bcae40b23432c0 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.255743 22107 tablet_replica.cc:333] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.256249 22107 raft_consensus.cc:2243] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.257040 22107 raft_consensus.cc:2272] T ca56a943ba904b779d7584f94c266344 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.259030 22107 tablet_replica.cc:333] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.259644 22107 raft_consensus.cc:2243] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.260208 22107 raft_consensus.cc:2272] T c192a749705545f298d61bbb8ccb5665 P 54640ab8abeb4c949426d66c7e5aed87 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.262174 22107 tablet_replica.cc:333] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.262698 22107 raft_consensus.cc:2243] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.263105 22107 raft_consensus.cc:2272] T 19d9facb5378417aaa3279b3dd9f02a9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.264881 22107 tablet_replica.cc:333] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.265407 22107 raft_consensus.cc:2243] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.266309 22107 raft_consensus.cc:2272] T ff06250da5d340babc2a28fd336380a7 P 54640ab8abeb4c949426d66c7e5aed87 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.268191 22107 tablet_replica.cc:333] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.268718 22107 raft_consensus.cc:2243] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.269675 22107 raft_consensus.cc:2272] T dbf1b9383e214ea7bb12a7f8784bffd1 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.271629 22107 tablet_replica.cc:333] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.272176 22107 raft_consensus.cc:2243] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.272714 22107 raft_consensus.cc:2272] T 8b87a4fcda7f4296b7f91f6ba3a14f29 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.274919 22107 tablet_replica.cc:333] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.275461 22107 raft_consensus.cc:2243] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.276438 22107 raft_consensus.cc:2272] T 71b8a974ecc34bd487fe9c208e6acc17 P 54640ab8abeb4c949426d66c7e5aed87 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.278612 22107 tablet_replica.cc:333] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.279163 22107 raft_consensus.cc:2243] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.279663 22107 raft_consensus.cc:2272] T 350816ae62fd4fb598a9ea6c84adf933 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.281508 22107 tablet_replica.cc:333] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.281991 22107 raft_consensus.cc:2243] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.282457 22107 raft_consensus.cc:2272] T f6161f3a92cf44138f9a68a6f005c4e9 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.284246 22107 tablet_replica.cc:333] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.284896 22107 raft_consensus.cc:2243] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.285759 22107 raft_consensus.cc:2272] T 29bc5916fff44ea3b1dcd31091d822b9 P 54640ab8abeb4c949426d66c7e5aed87 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.287503 22107 tablet_replica.cc:333] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.287953 22107 raft_consensus.cc:2243] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.288335 22107 raft_consensus.cc:2272] T 2984a5ac8b6b49c9aa8eb969a484d610 P 54640ab8abeb4c949426d66c7e5aed87 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.290385 22107 tablet_replica.cc:333] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87: stopping tablet replica
I20260709 12:16:49.290774 22107 raft_consensus.cc:2243] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:49.291153 22107 raft_consensus.cc:2272] T 19b18a93b6e8478e9f44ce88b342ab54 P 54640ab8abeb4c949426d66c7e5aed87 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.671859 22107 tablet_server.cc:196] TabletServer@127.21.150.196:0 shutdown complete.
I20260709 12:16:49.739446 22107 master.cc:562] Master@127.21.150.254:44861 shutting down...
I20260709 12:16:49.772545 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:49.773180 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:49.773532 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 259b912070e34dfba6c09de6bbb3d141: stopping tablet replica
I20260709 12:16:50.009667 22107 master.cc:584] Master@127.21.150.254:44861 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (242126 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:16:50.113137 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:39763
I20260709 12:16:50.114696 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:50.121459 31377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.121853 31378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.122356 31380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.122932 22107 server_base.cc:1061] running on GCE node
I20260709 12:16:50.125741 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:50.125943 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:50.126135 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599410126110 us; error 0 us; skew 500 ppm
I20260709 12:16:50.126734 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:50.129666 22107 webserver.cc:533] Webserver started at http://127.21.150.254:38869/ using document root <none> and password file <none>
I20260709 12:16:50.130200 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:50.130358 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:50.130622 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:50.132021 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "1f0c8e00074048c1901b985892084ace"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.136457 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:50.140029 31385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.140964 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:50.141341 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "1f0c8e00074048c1901b985892084ace"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.141681 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:50.154081 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:50.155365 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:50.194967 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:39763
I20260709 12:16:50.195163 31436 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:39763 every 8 connection(s)
I20260709 12:16:50.199158 31437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:50.209805 31437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace: Bootstrap starting.
I20260709 12:16:50.214766 31437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:50.219173 31437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace: No bootstrap required, opened a new log
I20260709 12:16:50.221521 31437 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER }
I20260709 12:16:50.221948 31437 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:50.222175 31437 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0c8e00074048c1901b985892084ace, State: Initialized, Role: FOLLOWER
I20260709 12:16:50.222837 31437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER }
I20260709 12:16:50.223316 31437 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:50.223526 31437 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:50.223839 31437 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.228509 31437 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER }
I20260709 12:16:50.229132 31437 leader_election.cc:304] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f0c8e00074048c1901b985892084ace; no voters: 
I20260709 12:16:50.230409 31437 leader_election.cc:290] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:50.230851 31440 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:50.232236 31440 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 1 LEADER]: Becoming Leader. State: Replica: 1f0c8e00074048c1901b985892084ace, State: Running, Role: LEADER
I20260709 12:16:50.232959 31440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER }
I20260709 12:16:50.233484 31437 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:50.237854 31441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f0c8e00074048c1901b985892084ace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER } }
I20260709 12:16:50.239521 31441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:50.238899 31442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f0c8e00074048c1901b985892084ace. Latest consensus state: current_term: 1 leader_uuid: "1f0c8e00074048c1901b985892084ace" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f0c8e00074048c1901b985892084ace" member_type: VOTER } }
I20260709 12:16:50.240502 31442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:50.242174 31448 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:50.250413 31448 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:50.251468 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:50.259897 31448 catalog_manager.cc:1383] Generated new cluster ID: 4ee1d4c72fc045eca8e625b9b060ae1c
I20260709 12:16:50.260246 31448 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:50.274248 31448 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:50.275590 31448 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:50.288771 31448 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace: Generated new TSK 0
I20260709 12:16:50.289515 31448 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:50.318246 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:50.324311 31458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.325270 31459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.329178 22107 server_base.cc:1061] running on GCE node
W20260709 12:16:50.329746 31461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.334039 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:50.334228 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:50.334438 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599410334398 us; error 0 us; skew 500 ppm
I20260709 12:16:50.335069 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:50.337338 22107 webserver.cc:533] Webserver started at http://127.21.150.193:44115/ using document root <none> and password file <none>
I20260709 12:16:50.337903 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:50.338109 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:50.338389 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:50.339440 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.343997 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:50.347252 31466 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.348083 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:16:50.348369 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.348670 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:50.367157 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:50.368371 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:50.369983 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:50.372243 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:50.372465 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.372716 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:50.372910 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.414408 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:37911
I20260709 12:16:50.414525 31528 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:37911 every 8 connection(s)
I20260709 12:16:50.419723 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:50.427600 31533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.428094 31534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.432189 31537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.432379 22107 server_base.cc:1061] running on GCE node
I20260709 12:16:50.434324 31529 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39763
I20260709 12:16:50.434752 31529 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:50.434955 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:50.435218 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:50.435425 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599410435407 us; error 0 us; skew 500 ppm
I20260709 12:16:50.435670 31529 heartbeater.cc:507] Master 127.21.150.254:39763 requested a full tablet report, sending...
I20260709 12:16:50.436244 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:50.438110 31402 ts_manager.cc:194] Registered new tserver with Master: 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:50.439128 22107 webserver.cc:533] Webserver started at http://127.21.150.194:46295/ using document root <none> and password file <none>
I20260709 12:16:50.439812 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:50.440049 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:50.440348 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:50.440661 31402 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39010
I20260709 12:16:50.441932 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "843512b2aa644b55a21b7e702943e02f"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.446892 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:16:50.450598 31541 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.451454 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:50.451870 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "843512b2aa644b55a21b7e702943e02f"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.452162 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:50.466519 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:50.467758 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:50.469208 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:50.471490 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:50.471719 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.471966 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:50.472146 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.513708 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:39965
I20260709 12:16:50.513828 31603 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:39965 every 8 connection(s)
I20260709 12:16:50.518594 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:50.526046 31607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.529304 31608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.529807 22107 server_base.cc:1061] running on GCE node
W20260709 12:16:50.530757 31610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.531985 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:50.532262 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:50.532513 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599410532493 us; error 0 us; skew 500 ppm
I20260709 12:16:50.533172 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:50.533407 31604 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39763
I20260709 12:16:50.533864 31604 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:50.534840 31604 heartbeater.cc:507] Master 127.21.150.254:39763 requested a full tablet report, sending...
I20260709 12:16:50.536186 22107 webserver.cc:533] Webserver started at http://127.21.150.195:40859/ using document root <none> and password file <none>
I20260709 12:16:50.536726 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:50.536936 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:50.537027 31402 ts_manager.cc:194] Registered new tserver with Master: 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:50.537235 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:50.538494 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "0bd0f6deb3b147509a008c574d9d879d"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.538607 31402 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39012
I20260709 12:16:50.543687 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 12:16:50.547560 31615 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.548316 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:16:50.548717 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "0bd0f6deb3b147509a008c574d9d879d"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.548980 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:50.563977 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:50.565186 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:50.566713 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:50.568872 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:50.569075 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.569346 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:50.569571 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:50.609997 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:38461
I20260709 12:16:50.610108 31677 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:38461 every 8 connection(s)
I20260709 12:16:50.614883 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:50.622188 31681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:50.624603 31682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.628070 31678 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39763
I20260709 12:16:50.628217 22107 server_base.cc:1061] running on GCE node
I20260709 12:16:50.628691 31678 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:50.628998 31685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:50.629736 31678 heartbeater.cc:507] Master 127.21.150.254:39763 requested a full tablet report, sending...
I20260709 12:16:50.629992 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:50.630255 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:50.630437 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599410630420 us; error 0 us; skew 500 ppm
I20260709 12:16:50.631160 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:50.632134 31402 ts_manager.cc:194] Registered new tserver with Master: 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:50.633899 22107 webserver.cc:533] Webserver started at http://127.21.150.196:35435/ using document root <none> and password file <none>
I20260709 12:16:50.634104 31402 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39018
I20260709 12:16:50.634568 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:50.634835 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:50.635160 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:50.636297 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-3-root/instance:
uuid: "bf0f149240cc4f27b97810c90eba7ff9"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.640532 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:50.643962 31689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.644806 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:16:50.645059 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
uuid: "bf0f149240cc4f27b97810c90eba7ff9"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:16:50.645298 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:50.666308 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:50.667518 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:50.668983 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:50.671214 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:50.671425 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.671675 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:50.671836 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:50.712566 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.196:40451
I20260709 12:16:50.712690 31751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.196:40451 every 8 connection(s)
I20260709 12:16:50.728338 31752 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39763
I20260709 12:16:50.728823 31752 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:50.729655 31752 heartbeater.cc:507] Master 127.21.150.254:39763 requested a full tablet report, sending...
I20260709 12:16:50.731662 31402 ts_manager.cc:194] Registered new tserver with Master: bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:50.732589 22107 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016638131s
I20260709 12:16:50.733335 31402 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39028
I20260709 12:16:50.761323 31402 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39042:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:16:51.015219 31643 tablet_service.cc:1511] Processing CreateTablet for tablet b52b6a2957cc45ae96342d2b42cf8ae1 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:51.017078 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52b6a2957cc45ae96342d2b42cf8ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.020289 31641 tablet_service.cc:1511] Processing CreateTablet for tablet fd72db62f85b4234aee25b5eda5abddd (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:51.022334 31642 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd06ecf6551463ba3c93bcf334bb6a9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:51.023276 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd72db62f85b4234aee25b5eda5abddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.023057 31717 tablet_service.cc:1511] Processing CreateTablet for tablet b52b6a2957cc45ae96342d2b42cf8ae1 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:51.023994 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd06ecf6551463ba3c93bcf334bb6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.024348 31717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52b6a2957cc45ae96342d2b42cf8ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.026283 31569 tablet_service.cc:1511] Processing CreateTablet for tablet b52b6a2957cc45ae96342d2b42cf8ae1 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:16:51.027647 31569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b52b6a2957cc45ae96342d2b42cf8ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.030709 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd06ecf6551463ba3c93bcf334bb6a9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:51.031193 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 096e1ab4f2704ddaaad535361f46d6a2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:51.032202 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd06ecf6551463ba3c93bcf334bb6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.032608 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096e1ab4f2704ddaaad535361f46d6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.034674 31567 tablet_service.cc:1511] Processing CreateTablet for tablet 096e1ab4f2704ddaaad535361f46d6a2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:51.036048 31567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096e1ab4f2704ddaaad535361f46d6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.074043 31566 tablet_service.cc:1511] Processing CreateTablet for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:51.075650 31566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.081104 31643 tablet_service.cc:1511] Processing CreateTablet for tablet af22442d6a9141f2a4607ba57cf4e0ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:51.081961 31642 tablet_service.cc:1511] Processing CreateTablet for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:51.082650 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af22442d6a9141f2a4607ba57cf4e0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.083391 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.089779 31714 tablet_service.cc:1511] Processing CreateTablet for tablet 6019e17e78a14e9fa70a4f5aa6e26056 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:51.091439 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6019e17e78a14e9fa70a4f5aa6e26056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.087165 31715 tablet_service.cc:1511] Processing CreateTablet for tablet fd72db62f85b4234aee25b5eda5abddd (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:51.093381 31564 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0f4df696784a248aaa0b98db69dfa0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:51.094369 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd72db62f85b4234aee25b5eda5abddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.094844 31564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0f4df696784a248aaa0b98db69dfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.088006 31565 tablet_service.cc:1511] Processing CreateTablet for tablet 6019e17e78a14e9fa70a4f5aa6e26056 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:51.098397 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6019e17e78a14e9fa70a4f5aa6e26056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.102185 31771 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:51.104949 31640 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0f4df696784a248aaa0b98db69dfa0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:51.106356 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0f4df696784a248aaa0b98db69dfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.115947 31771 tablet_bootstrap.cc:654] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.115705 31494 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd06ecf6551463ba3c93bcf334bb6a9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:16:51.117542 31772 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:51.118806 31493 tablet_service.cc:1511] Processing CreateTablet for tablet 096e1ab4f2704ddaaad535361f46d6a2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:16:51.121284 31492 tablet_service.cc:1511] Processing CreateTablet for tablet fd72db62f85b4234aee25b5eda5abddd (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:16:51.123675 31491 tablet_service.cc:1511] Processing CreateTablet for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:16:51.125912 31490 tablet_service.cc:1511] Processing CreateTablet for tablet 6019e17e78a14e9fa70a4f5aa6e26056 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:16:51.117280 31494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd06ecf6551463ba3c93bcf334bb6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.129914 31489 tablet_service.cc:1511] Processing CreateTablet for tablet af22442d6a9141f2a4607ba57cf4e0ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:51.130506 31563 tablet_service.cc:1511] Processing CreateTablet for tablet c506263e3b0d4ce98f8336aec8159188 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:51.131419 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af22442d6a9141f2a4607ba57cf4e0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.131875 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506263e3b0d4ce98f8336aec8159188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.133862 31491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f174edb7d6fa49b4a8d02fcdae3fdbe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.134872 31490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6019e17e78a14e9fa70a4f5aa6e26056. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.138926 31713 tablet_service.cc:1511] Processing CreateTablet for tablet af22442d6a9141f2a4607ba57cf4e0ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:16:51.139655 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 096e1ab4f2704ddaaad535361f46d6a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.140348 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af22442d6a9141f2a4607ba57cf4e0ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.145665 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd72db62f85b4234aee25b5eda5abddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.143388 31712 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0f4df696784a248aaa0b98db69dfa0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:16:51.149024 31712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0f4df696784a248aaa0b98db69dfa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.151852 31772 tablet_bootstrap.cc:654] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.163828 31488 tablet_service.cc:1511] Processing CreateTablet for tablet c506263e3b0d4ce98f8336aec8159188 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:51.165337 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506263e3b0d4ce98f8336aec8159188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.192237 31641 tablet_service.cc:1511] Processing CreateTablet for tablet c506263e3b0d4ce98f8336aec8159188 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:16:51.193843 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c506263e3b0d4ce98f8336aec8159188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.217208 31774 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:51.238561 31714 tablet_service.cc:1511] Processing CreateTablet for tablet b667dda9be9d46cdba95860c4c37102e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:51.239979 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b667dda9be9d46cdba95860c4c37102e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.242920 31774 tablet_bootstrap.cc:654] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:51.245702 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:51.247560 31489 tablet_service.cc:1511] Processing CreateTablet for tablet b667dda9be9d46cdba95860c4c37102e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:51.249161 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b667dda9be9d46cdba95860c4c37102e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.251711 31771 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:51.254685 31771 ts_tablet_manager.cc:1403] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.153s	user 0.018s	sys 0.024s
I20260709 12:16:51.254683 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 28ff8e905a7b4eb5b2cea1df313efa14 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:51.256748 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ff8e905a7b4eb5b2cea1df313efa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.258460 31771 raft_consensus.cc:359] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.259241 31771 raft_consensus.cc:385] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.259526 31771 raft_consensus.cc:740] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.260257 31771 consensus_queue.cc:260] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.263867 31772 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:51.264362 31772 ts_tablet_manager.cc:1403] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.147s	user 0.020s	sys 0.025s
I20260709 12:16:51.267184 31772 raft_consensus.cc:359] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.267752 31772 raft_consensus.cc:385] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.268512 31772 raft_consensus.cc:740] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.269626 31772 consensus_queue.cc:260] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.275578 31776 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:51.284957 31776 tablet_bootstrap.cc:654] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.285120 31604 heartbeater.cc:499] Master 127.21.150.254:39763 was elected leader, sending a full tablet report...
I20260709 12:16:51.286276 31772 ts_tablet_manager.cc:1434] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 12:16:51.290040 31565 tablet_service.cc:1511] Processing CreateTablet for tablet b667dda9be9d46cdba95860c4c37102e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:16:51.291427 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b667dda9be9d46cdba95860c4c37102e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.291936 31772 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:51.292021 31563 tablet_service.cc:1511] Processing CreateTablet for tablet 75b15073c4e54ef1af602f9c83702642 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:51.301523 31712 tablet_service.cc:1511] Processing CreateTablet for tablet 28ff8e905a7b4eb5b2cea1df313efa14 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:51.302848 31712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ff8e905a7b4eb5b2cea1df313efa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.300109 31640 tablet_service.cc:1511] Processing CreateTablet for tablet 75b15073c4e54ef1af602f9c83702642 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:51.307029 31771 ts_tablet_manager.cc:1434] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.051s	user 0.012s	sys 0.017s
I20260709 12:16:51.307996 31678 heartbeater.cc:499] Master 127.21.150.254:39763 was elected leader, sending a full tablet report...
I20260709 12:16:51.308143 31771 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:51.310424 31492 tablet_service.cc:1511] Processing CreateTablet for tablet 28ff8e905a7b4eb5b2cea1df313efa14 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:16:51.310833 31493 tablet_service.cc:1511] Processing CreateTablet for tablet 33acdf64b87145ab8a38a66147ae23b5 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:51.311856 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28ff8e905a7b4eb5b2cea1df313efa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.313539 31713 tablet_service.cc:1511] Processing CreateTablet for tablet 75b15073c4e54ef1af602f9c83702642 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:16:51.314908 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b15073c4e54ef1af602f9c83702642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.316614 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 33acdf64b87145ab8a38a66147ae23b5 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:51.306725 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 33acdf64b87145ab8a38a66147ae23b5 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:16:51.305462 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b15073c4e54ef1af602f9c83702642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.318495 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33acdf64b87145ab8a38a66147ae23b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.319164 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33acdf64b87145ab8a38a66147ae23b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.322544 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b15073c4e54ef1af602f9c83702642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.324792 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33acdf64b87145ab8a38a66147ae23b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.338423 31715 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee706eb478f44e0979857553c460e94 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:51.339792 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee706eb478f44e0979857553c460e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.352365 31771 tablet_bootstrap.cc:654] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.358214 31488 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee706eb478f44e0979857553c460e94 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:51.359565 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee706eb478f44e0979857553c460e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.360445 31774 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:51.361775 31774 ts_tablet_manager.cc:1403] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.152s	user 0.026s	sys 0.010s
W20260709 12:16:51.364420 31679 tablet.cc:2406] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:51.372623 31774 raft_consensus.cc:359] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.374658 31774 raft_consensus.cc:385] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.385943 31774 raft_consensus.cc:740] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.386215 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 9ee706eb478f44e0979857553c460e94 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:16:51.387642 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ee706eb478f44e0979857553c460e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.387216 31774 consensus_queue.cc:260] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.394987 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:51.396649 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.422459 31640 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:51.423863 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.424602 31491 tablet_service.cc:1511] Processing CreateTablet for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:16:51.426098 31564 tablet_service.cc:1511] Processing CreateTablet for tablet 74bd4922c2764afd8737751cb3d17cb8 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:51.426692 31491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c6ef1dbe6184d41b2a5edfb45c8315e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.427368 31564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bd4922c2764afd8737751cb3d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.434058 31772 tablet_bootstrap.cc:654] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.438542 31774 ts_tablet_manager.cc:1434] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.001s
I20260709 12:16:51.440322 31752 heartbeater.cc:499] Master 127.21.150.254:39763 was elected leader, sending a full tablet report...
I20260709 12:16:51.442301 31774 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:51.443797 31529 heartbeater.cc:499] Master 127.21.150.254:39763 was elected leader, sending a full tablet report...
I20260709 12:16:51.443912 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 74bd4922c2764afd8737751cb3d17cb8 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:51.444376 31715 tablet_service.cc:1511] Processing CreateTablet for tablet 74bd4922c2764afd8737751cb3d17cb8 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:16:51.445694 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bd4922c2764afd8737751cb3d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.446183 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 74bd4922c2764afd8737751cb3d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.454931 31492 tablet_service.cc:1511] Processing CreateTablet for tablet e5dfd0d3d6f941e68232bcc32722b5de (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:51.471357 31771 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:51.473785 31771 ts_tablet_manager.cc:1403] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.166s	user 0.035s	sys 0.048s
I20260709 12:16:51.477883 31771 raft_consensus.cc:359] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.478475 31771 raft_consensus.cc:385] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.478731 31771 raft_consensus.cc:740] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.479442 31771 consensus_queue.cc:260] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.472659 31774 tablet_bootstrap.cc:654] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.485865 31776 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:51.483270 31771 ts_tablet_manager.cc:1434] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.003s
I20260709 12:16:51.486810 31776 ts_tablet_manager.cc:1403] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.212s	user 0.031s	sys 0.015s
I20260709 12:16:51.490720 31776 raft_consensus.cc:359] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.491398 31776 raft_consensus.cc:385] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.491676 31776 raft_consensus.cc:740] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.492414 31776 consensus_queue.cc:260] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.496755 31771 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:51.496812 31776 ts_tablet_manager.cc:1434] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:51.503587 31771 tablet_bootstrap.cc:654] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.506271 31774 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:51.506856 31774 ts_tablet_manager.cc:1403] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.065s	user 0.003s	sys 0.008s
I20260709 12:16:51.509924 31774 raft_consensus.cc:359] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.510648 31774 raft_consensus.cc:385] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.510952 31774 raft_consensus.cc:740] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.511817 31774 consensus_queue.cc:260] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.515000 31714 tablet_service.cc:1511] Processing CreateTablet for tablet ecff280f5652462db2e6473ea7424a2f (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:51.516089 31564 tablet_service.cc:1511] Processing CreateTablet for tablet e5dfd0d3d6f941e68232bcc32722b5de (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:51.516934 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecff280f5652462db2e6473ea7424a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.517390 31564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5dfd0d3d6f941e68232bcc32722b5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.519369 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5dfd0d3d6f941e68232bcc32722b5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.540511 31772 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:51.541165 31772 ts_tablet_manager.cc:1403] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.251s	user 0.027s	sys 0.002s
I20260709 12:16:51.540679 31489 tablet_service.cc:1511] Processing CreateTablet for tablet ecff280f5652462db2e6473ea7424a2f (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:51.550505 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecff280f5652462db2e6473ea7424a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.560329 31772 raft_consensus.cc:359] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.561172 31772 raft_consensus.cc:385] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.561501 31772 raft_consensus.cc:740] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.564436 31771 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:51.564988 31771 ts_tablet_manager.cc:1403] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.021s
I20260709 12:16:51.542819 31776 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:51.541612 31643 tablet_service.cc:1511] Processing CreateTablet for tablet e5dfd0d3d6f941e68232bcc32722b5de (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:16:51.567022 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5dfd0d3d6f941e68232bcc32722b5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.568111 31771 raft_consensus.cc:359] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.568922 31771 raft_consensus.cc:385] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.569232 31771 raft_consensus.cc:740] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.570096 31771 consensus_queue.cc:260] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.572326 31776 tablet_bootstrap.cc:654] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.572383 31771 ts_tablet_manager.cc:1434] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:16:51.573776 31772 consensus_queue.cc:260] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.576169 31772 ts_tablet_manager.cc:1434] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 12:16:51.577179 31772 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:51.543167 31565 tablet_service.cc:1511] Processing CreateTablet for tablet ecff280f5652462db2e6473ea7424a2f (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:16:51.544262 31774 ts_tablet_manager.cc:1434] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.003s
I20260709 12:16:51.544958 31641 tablet_service.cc:1511] Processing CreateTablet for tablet dc169af544e14e668b85ec4da6ca648d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:51.578208 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecff280f5652462db2e6473ea7424a2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.578357 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc169af544e14e668b85ec4da6ca648d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.581703 31771 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:51.578411 31774 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:51.588192 31771 tablet_bootstrap.cc:654] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.588845 31774 tablet_bootstrap.cc:654] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.592115 31772 tablet_bootstrap.cc:654] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.599859 31715 tablet_service.cc:1511] Processing CreateTablet for tablet dc169af544e14e668b85ec4da6ca648d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:51.601769 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc169af544e14e668b85ec4da6ca648d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.622131 31489 tablet_service.cc:1511] Processing CreateTablet for tablet dc169af544e14e668b85ec4da6ca648d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:16:51.624116 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc169af544e14e668b85ec4da6ca648d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.624797 31564 tablet_service.cc:1511] Processing CreateTablet for tablet a6c957e6781843b9b69e2d98053daa74 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:51.626188 31564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c957e6781843b9b69e2d98053daa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.626855 31782 raft_consensus.cc:493] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.627569 31782 raft_consensus.cc:515] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.630494 31640 tablet_service.cc:1511] Processing CreateTablet for tablet a6c957e6781843b9b69e2d98053daa74 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:51.639740 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c957e6781843b9b69e2d98053daa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.642109 31782 leader_election.cc:290] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:51.647446 31779 raft_consensus.cc:493] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.661753 31778 raft_consensus.cc:493] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.661509 31714 tablet_service.cc:1511] Processing CreateTablet for tablet a6c957e6781843b9b69e2d98053daa74 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:16:51.662292 31778 raft_consensus.cc:515] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.662979 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6c957e6781843b9b69e2d98053daa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.665046 31778 leader_election.cc:290] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:51.683648 31774 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:51.683859 31772 tablet_bootstrap.cc:492] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:51.684245 31774 ts_tablet_manager.cc:1403] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.106s	user 0.001s	sys 0.011s
I20260709 12:16:51.684288 31772 ts_tablet_manager.cc:1403] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.107s	user 0.019s	sys 0.023s
I20260709 12:16:51.655907 31779 raft_consensus.cc:515] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.689388 31779 leader_election.cc:290] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:51.693869 31778 raft_consensus.cc:493] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.707212 31492 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5f0372a71a4abe9b70c3772934163c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:51.708650 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5f0372a71a4abe9b70c3772934163c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.708748 31565 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5f0372a71a4abe9b70c3772934163c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:51.709985 31771 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:51.710158 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5f0372a71a4abe9b70c3772934163c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.710542 31771 ts_tablet_manager.cc:1403] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.129s	user 0.020s	sys 0.006s
I20260709 12:16:51.700081 31778 raft_consensus.cc:515] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.714957 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5f0372a71a4abe9b70c3772934163c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:16:51.716285 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5f0372a71a4abe9b70c3772934163c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.716041 31713 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfd13293d3149f6a1ca73fd10852083 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:51.717042 31778 leader_election.cc:290] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:51.717319 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfd13293d3149f6a1ca73fd10852083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.718520 31782 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.719041 31782 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.691107 31772 raft_consensus.cc:359] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.732622 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfd13293d3149f6a1ca73fd10852083 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:51.734752 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfd13293d3149f6a1ca73fd10852083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.721355 31782 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:51.726439 31772 raft_consensus.cc:385] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.777663 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 6d00a005dd8945d9acac25d9822223a6 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:51.779660 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d00a005dd8945d9acac25d9822223a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.794708 31563 tablet_service.cc:1511] Processing CreateTablet for tablet 3c241197f69848f4b2142121633ec9ef (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:51.796696 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c241197f69848f4b2142121633ec9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.806250 31771 raft_consensus.cc:359] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.807267 31771 raft_consensus.cc:385] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.807670 31771 raft_consensus.cc:740] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.759235 31772 raft_consensus.cc:740] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.818651 31776 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:51.818840 31785 raft_consensus.cc:493] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.819252 31776 ts_tablet_manager.cc:1403] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.278s	user 0.025s	sys 0.024s
I20260709 12:16:51.819401 31785 raft_consensus.cc:515] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.821897 31772 consensus_queue.cc:260] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.822414 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:51.823141 31727 raft_consensus.cc:2468] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:51.824321 31772 ts_tablet_manager.cc:1434] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.001s
I20260709 12:16:51.822625 31785 leader_election.cc:290] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:51.824608 31616 leader_election.cc:304] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:51.825397 31772 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:51.831277 31641 tablet_service.cc:1511] Processing CreateTablet for tablet 6d00a005dd8945d9acac25d9822223a6 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:51.832607 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d00a005dd8945d9acac25d9822223a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.833524 31776 raft_consensus.cc:359] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:51.834254 31776 raft_consensus.cc:385] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.834540 31776 raft_consensus.cc:740] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.835443 31776 consensus_queue.cc:260] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:51.825558 31778 raft_consensus.cc:2804] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.837677 31778 raft_consensus.cc:493] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.838048 31778 raft_consensus.cc:3060] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.808729 31771 consensus_queue.cc:260] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:51.857322 31491 tablet_service.cc:1511] Processing CreateTablet for tablet 6d00a005dd8945d9acac25d9822223a6 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:16:51.858738 31491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d00a005dd8945d9acac25d9822223a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.863288 31569 tablet_service.cc:1511] Processing CreateTablet for tablet e4ba9fa8e8004145a227359512e2443d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:51.863823 31715 tablet_service.cc:1511] Processing CreateTablet for tablet 3c241197f69848f4b2142121633ec9ef (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:51.863287 31489 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfd13293d3149f6a1ca73fd10852083 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:16:51.865195 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c241197f69848f4b2142121633ec9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.865419 31774 raft_consensus.cc:359] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.866060 31569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4ba9fa8e8004145a227359512e2443d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.874178 31774 raft_consensus.cc:385] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:51.874570 31774 raft_consensus.cc:740] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:51.875325 31774 consensus_queue.cc:260] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:51.865486 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfd13293d3149f6a1ca73fd10852083. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.889259 31712 tablet_service.cc:1511] Processing CreateTablet for tablet 59dc1ab82c334d8f8954116a598b7c8b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:51.890771 31712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59dc1ab82c334d8f8954116a598b7c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.873946 31567 tablet_service.cc:1511] Processing CreateTablet for tablet 59dc1ab82c334d8f8954116a598b7c8b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:51.891984 31639 tablet_service.cc:1511] Processing CreateTablet for tablet e4ba9fa8e8004145a227359512e2443d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:51.892500 31567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59dc1ab82c334d8f8954116a598b7c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.893961 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:51.896030 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:51.896776 31579 raft_consensus.cc:2468] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:51.879926 31774 ts_tablet_manager.cc:1434] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.195s	user 0.009s	sys 0.000s
I20260709 12:16:51.897583 31776 ts_tablet_manager.cc:1434] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 12:16:51.898101 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:51.898648 31774 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:51.898708 31776 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:51.899914 31691 leader_election.cc:304] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
W20260709 12:16:51.900909 31691 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.901660 31782 raft_consensus.cc:2804] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:51.902077 31782 raft_consensus.cc:493] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:51.902462 31782 raft_consensus.cc:3060] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.866864 31785 raft_consensus.cc:493] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:51.904040 31785 raft_consensus.cc:515] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:51.906543 31785 leader_election.cc:290] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:51.907913 31772 tablet_bootstrap.cc:654] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.919034 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:51.893373 31639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4ba9fa8e8004145a227359512e2443d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.919826 31578 raft_consensus.cc:2468] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:51.919814 31778 raft_consensus.cc:515] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:51.923094 31653 raft_consensus.cc:2393] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:51.890637 31642 tablet_service.cc:1511] Processing CreateTablet for tablet 3c241197f69848f4b2142121633ec9ef (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:16:51.930665 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c241197f69848f4b2142121633ec9ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.933871 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:51.957401 31617 leader_election.cc:343] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:51.958696 31638 tablet_service.cc:1511] Processing CreateTablet for tablet 5590853f22ef45de955f77e7648a574b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:51.959843 31774 tablet_bootstrap.cc:654] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:51.976214 31638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5590853f22ef45de955f77e7648a574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.977784 31493 tablet_service.cc:1511] Processing CreateTablet for tablet e4ba9fa8e8004145a227359512e2443d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:16:51.979161 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4ba9fa8e8004145a227359512e2443d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:51.990149 31771 ts_tablet_manager.cc:1434] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.279s	user 0.015s	sys 0.018s
I20260709 12:16:52.004824 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:52.005561 31579 raft_consensus.cc:3060] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:51.974855 31782 raft_consensus.cc:515] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.005930 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:52.021715 31727 raft_consensus.cc:2468] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:52.023842 31542 leader_election.cc:304] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:52.016935 31779 raft_consensus.cc:493] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.027359 31779 raft_consensus.cc:515] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.031352 31492 tablet_service.cc:1511] Processing CreateTablet for tablet 59dc1ab82c334d8f8954116a598b7c8b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:16:52.033214 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59dc1ab82c334d8f8954116a598b7c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.044205 31714 tablet_service.cc:1511] Processing CreateTablet for tablet a194f9fff2f94f8d8ecfe700ee83d998 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:52.045076 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:52.045691 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a194f9fff2f94f8d8ecfe700ee83d998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.045948 31778 leader_election.cc:290] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.046564 31779 leader_election.cc:290] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:52.047183 31779 raft_consensus.cc:2804] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.047562 31779 raft_consensus.cc:493] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.047945 31779 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.044389 31776 tablet_bootstrap.cc:654] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.051405 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.053541 31806 raft_consensus.cc:493] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.054231 31806 raft_consensus.cc:515] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.054440 31488 tablet_service.cc:1511] Processing CreateTablet for tablet 5590853f22ef45de955f77e7648a574b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:52.055827 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5590853f22ef45de955f77e7648a574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.056646 31806 leader_election.cc:290] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.062507 31771 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:52.069056 31563 tablet_service.cc:1511] Processing CreateTablet for tablet a194f9fff2f94f8d8ecfe700ee83d998 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:52.070618 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a194f9fff2f94f8d8ecfe700ee83d998. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.106426 31542 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 80823.6us
I20260709 12:16:52.112730 31727 raft_consensus.cc:2393] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:52.132679 31643 tablet_service.cc:1511] Processing CreateTablet for tablet a194f9fff2f94f8d8ecfe700ee83d998 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:16:52.141898 31782 leader_election.cc:290] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:52.145294 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
W20260709 12:16:52.157557 31616 leader_election.cc:343] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.157882 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b52b6a2957cc45ae96342d2b42cf8ae1" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:52.158088 31616 leader_election.cc:304] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:52.160045 31806 raft_consensus.cc:2749] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.159919 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:52.158787 31653 raft_consensus.cc:2393] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:52.165678 31579 raft_consensus.cc:2468] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:52.166076 31494 tablet_service.cc:1511] Processing CreateTablet for tablet aeff768d3659498c9e632d68b32a28d2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:52.163766 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.171339 31617 leader_election.cc:304] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:52.172708 31806 raft_consensus.cc:2804] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:52.172437 31713 tablet_service.cc:1511] Processing CreateTablet for tablet 5590853f22ef45de955f77e7648a574b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:16:52.173295 31806 raft_consensus.cc:697] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:52.173835 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5590853f22ef45de955f77e7648a574b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.173835 31494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff768d3659498c9e632d68b32a28d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.174212 31806 consensus_queue.cc:237] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.164487 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a194f9fff2f94f8d8ecfe700ee83d998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.172767 31693 leader_election.cc:304] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:52.177946 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.178534 31782 raft_consensus.cc:2749] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:52.179574 31690 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.180024 31690 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:52.183887 31779 raft_consensus.cc:515] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.185089 31567 tablet_service.cc:1511] Processing CreateTablet for tablet aeff768d3659498c9e632d68b32a28d2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:52.181267 31771 tablet_bootstrap.cc:654] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:52.210525 31616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 53008.5us
I20260709 12:16:52.225018 31772 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:52.225728 31772 ts_tablet_manager.cc:1403] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.401s	user 0.041s	sys 0.050s
I20260709 12:16:52.226334 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:52.226989 31727 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.229130 31772 raft_consensus.cc:359] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.230016 31772 raft_consensus.cc:385] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.230381 31772 raft_consensus.cc:740] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.231099 31640 tablet_service.cc:1511] Processing CreateTablet for tablet aeff768d3659498c9e632d68b32a28d2 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:16:52.233877 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.236927 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 3f18f4e3f28a4fa7844c90b02bea7476 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:52.237555 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:52.238142 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff768d3659498c9e632d68b32a28d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.238209 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f18f4e3f28a4fa7844c90b02bea7476. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.239423 31468 leader_election.cc:343] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:52.248296 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:16:52.232050 31616 leader_election.cc:343] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.231416 31782 raft_consensus.cc:2749] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.268637 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.269096 31776 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:52.269634 31776 ts_tablet_manager.cc:1403] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.371s	user 0.008s	sys 0.018s
I20260709 12:16:52.271618 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:52.271553 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:52.272692 31776 raft_consensus.cc:359] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.234819 31779 leader_election.cc:290] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.262207 31772 consensus_queue.cc:260] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.264668 31727 raft_consensus.cc:2468] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:52.273412 31776 raft_consensus.cc:385] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.283587 31776 raft_consensus.cc:740] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.284372 31776 consensus_queue.cc:260] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.285295 31774 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:52.285812 31774 ts_tablet_manager.cc:1403] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.387s	user 0.018s	sys 0.011s
W20260709 12:16:52.285101 31616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 50755.3us
I20260709 12:16:52.286506 31776 ts_tablet_manager.cc:1434] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.017s	user 0.002s	sys 0.004s
I20260709 12:16:52.287508 31776 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:52.276016 31567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeff768d3659498c9e632d68b32a28d2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.276461 31542 leader_election.cc:343] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.276081 31494 tablet_service.cc:1511] Processing CreateTablet for tablet 3f18f4e3f28a4fa7844c90b02bea7476 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
W20260709 12:16:52.291507 31542 leader_election.cc:343] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.291901 31494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f18f4e3f28a4fa7844c90b02bea7476. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.292171 31542 leader_election.cc:343] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.288935 31774 raft_consensus.cc:359] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
W20260709 12:16:52.278970 31467 leader_election.cc:343] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.293248 31542 leader_election.cc:304] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:52.293771 31774 raft_consensus.cc:385] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.300289 31774 raft_consensus.cc:740] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.300776 31776 tablet_bootstrap.cc:654] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.302040 31774 consensus_queue.cc:260] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.310452 31638 tablet_service.cc:1511] Processing CreateTablet for tablet dc5b5e283ec34f03b184be6dfa4e046e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:52.310938 31569 tablet_service.cc:1511] Processing CreateTablet for tablet 3f18f4e3f28a4fa7844c90b02bea7476 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:16:52.311738 31774 ts_tablet_manager.cc:1434] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:16:52.311908 31638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5b5e283ec34f03b184be6dfa4e046e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.312367 31569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f18f4e3f28a4fa7844c90b02bea7476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.303085 31779 raft_consensus.cc:2804] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:52.313961 31779 raft_consensus.cc:697] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:52.280391 31772 ts_tablet_manager.cc:1434] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 12:16:52.319559 31772 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:52.314844 31779 consensus_queue.cc:237] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.321589 31488 tablet_service.cc:1511] Processing CreateTablet for tablet dc5b5e283ec34f03b184be6dfa4e046e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:52.323078 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5b5e283ec34f03b184be6dfa4e046e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.326638 31774 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:52.305785 31806 raft_consensus.cc:493] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.358152 31806 raft_consensus.cc:515] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.331254 31565 tablet_service.cc:1511] Processing CreateTablet for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:52.360412 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.361330 31491 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad0edffb3874e40994f98fd888a00b3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:52.362110 31772 tablet_bootstrap.cc:654] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.362664 31491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad0edffb3874e40994f98fd888a00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.362608 31711 tablet_service.cc:1511] Processing CreateTablet for tablet a44cca173ddc49d99e53c40bccf8b5c9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:52.364033 31711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44cca173ddc49d99e53c40bccf8b5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.332864 31639 tablet_service.cc:1511] Processing CreateTablet for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:52.333549 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:16:52.337486 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 1937adc6d4be49a0aa4d15f000842da3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:52.370214 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1937adc6d4be49a0aa4d15f000842da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.370862 31776 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:52.371531 31776 ts_tablet_manager.cc:1403] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.084s	user 0.017s	sys 0.009s
I20260709 12:16:52.335397 31493 tablet_service.cc:1511] Processing CreateTablet for tablet 1937adc6d4be49a0aa4d15f000842da3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:52.338820 31640 tablet_service.cc:1511] Processing CreateTablet for tablet 1937adc6d4be49a0aa4d15f000842da3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:16:52.340937 31714 tablet_service.cc:1511] Processing CreateTablet for tablet 506b4da2b0be46d5bf3ff52167d4419b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:52.342603 31492 tablet_service.cc:1511] Processing CreateTablet for tablet 506b4da2b0be46d5bf3ff52167d4419b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:52.321316 31712 tablet_service.cc:1511] Processing CreateTablet for tablet dc5b5e283ec34f03b184be6dfa4e046e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:16:52.343333 31774 tablet_bootstrap.cc:654] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.345337 31564 tablet_service.cc:1511] Processing CreateTablet for tablet 506b4da2b0be46d5bf3ff52167d4419b (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:16:52.382531 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:52.383036 31564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506b4da2b0be46d5bf3ff52167d4419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.383189 31579 raft_consensus.cc:2468] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:52.383606 31567 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad0edffb3874e40994f98fd888a00b3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:52.384866 31567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad0edffb3874e40994f98fd888a00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.385027 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506b4da2b0be46d5bf3ff52167d4419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.345891 31771 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:52.347898 31641 tablet_service.cc:1511] Processing CreateTablet for tablet 098e97adee224f86ba1bfa07ce392b68 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:52.385862 31771 ts_tablet_manager.cc:1403] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.324s	user 0.023s	sys 0.038s
I20260709 12:16:52.349934 31717 tablet_service.cc:1511] Processing CreateTablet for tablet 098e97adee224f86ba1bfa07ce392b68 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:52.386926 31717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098e97adee224f86ba1bfa07ce392b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.387323 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506b4da2b0be46d5bf3ff52167d4419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.350844 31489 tablet_service.cc:1511] Processing CreateTablet for tablet 098e97adee224f86ba1bfa07ce392b68 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:16:52.390101 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.390290 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098e97adee224f86ba1bfa07ce392b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.392225 31712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5b5e283ec34f03b184be6dfa4e046e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.354435 31566 tablet_service.cc:1511] Processing CreateTablet for tablet a44cca173ddc49d99e53c40bccf8b5c9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:52.354449 31642 tablet_service.cc:1511] Processing CreateTablet for tablet a44cca173ddc49d99e53c40bccf8b5c9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:16:52.394635 31566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44cca173ddc49d99e53c40bccf8b5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.398790 31617 leader_election.cc:304] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
W20260709 12:16:52.357558 31616 leader_election.cc:343] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.399569 31616 leader_election.cc:304] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:52.367947 31806 leader_election.cc:290] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:52.399746 31778 raft_consensus.cc:2804] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.366297 31639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ac1bb8ec79464ebaa8b8d0d4a5747d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.389119 31771 raft_consensus.cc:359] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.401825 31778 raft_consensus.cc:493] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.403791 31808 raft_consensus.cc:2749] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.404109 31778 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.403911 31771 raft_consensus.cc:385] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.408106 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 6ad0edffb3874e40994f98fd888a00b3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:16:52.409467 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ad0edffb3874e40994f98fd888a00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.412786 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a44cca173ddc49d99e53c40bccf8b5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.409102 31771 raft_consensus.cc:740] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.426065 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1937adc6d4be49a0aa4d15f000842da3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.413326 31616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 55815us
I20260709 12:16:52.429004 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1937adc6d4be49a0aa4d15f000842da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.439121 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 098e97adee224f86ba1bfa07ce392b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.447480 31490 tablet_service.cc:1511] Processing CreateTablet for tablet 384f88a7446842bfb4a6c71e921c9049 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:52.448868 31490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 384f88a7446842bfb4a6c71e921c9049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.450520 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.452728 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:52.452884 31715 tablet_service.cc:1511] Processing CreateTablet for tablet a2cd62a9ae5d4430bdb3799b63ca2137 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
W20260709 12:16:52.453410 31616 leader_election.cc:343] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.454334 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2cd62a9ae5d4430bdb3799b63ca2137. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.454769 31470 leader_election.cc:343] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.454936 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:52.454147 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:52.455250 31470 leader_election.cc:304] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:52.456525 31713 tablet_service.cc:1511] Processing CreateTablet for tablet 384f88a7446842bfb4a6c71e921c9049 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:52.457113 31807 raft_consensus.cc:2749] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.439214 31771 consensus_queue.cc:260] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.458177 31637 tablet_service.cc:1511] Processing CreateTablet for tablet a2cd62a9ae5d4430bdb3799b63ca2137 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:52.450187 31562 tablet_service.cc:1511] Processing CreateTablet for tablet 384f88a7446842bfb4a6c71e921c9049 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:16:52.459826 31771 ts_tablet_manager.cc:1434] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 12:16:52.460122 31562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 384f88a7446842bfb4a6c71e921c9049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.460882 31771 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:52.460182 31399 catalog_manager.cc:5697] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:52.463428 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 384f88a7446842bfb4a6c71e921c9049. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.471066 31470 leader_election.cc:343] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.479177 31470 leader_election.cc:304] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:52.489190 31807 raft_consensus.cc:2749] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.491127 31488 tablet_service.cc:1511] Processing CreateTablet for tablet a2cd62a9ae5d4430bdb3799b63ca2137 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:16:52.491889 31776 raft_consensus.cc:359] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.492631 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2cd62a9ae5d4430bdb3799b63ca2137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.496174 31643 tablet_service.cc:1511] Processing CreateTablet for tablet e302635e586c48819cf549549c5f4446 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:52.473402 31637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2cd62a9ae5d4430bdb3799b63ca2137. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.475440 31778 raft_consensus.cc:515] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.492650 31776 raft_consensus.cc:385] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.507167 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e302635e586c48819cf549549c5f4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.511987 31651 raft_consensus.cc:2393] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:52.516505 31400 catalog_manager.cc:5697] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:52.518724 31636 tablet_service.cc:1511] Processing CreateTablet for tablet 9da10e70d33e4495babdca0057997818 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:52.520088 31636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da10e70d33e4495babdca0057997818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.520794 31778 leader_election.cc:290] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:52.526573 31487 tablet_service.cc:1511] Processing CreateTablet for tablet 9da10e70d33e4495babdca0057997818 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:52.528055 31487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da10e70d33e4495babdca0057997818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.530058 31710 tablet_service.cc:1511] Processing CreateTablet for tablet e302635e586c48819cf549549c5f4446 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:52.531468 31710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e302635e586c48819cf549549c5f4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.534025 31563 tablet_service.cc:1511] Processing CreateTablet for tablet e302635e586c48819cf549549c5f4446 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:16:52.534332 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:52.535023 31579 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.535426 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e302635e586c48819cf549549c5f4446. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.505131 31776 raft_consensus.cc:740] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.542366 31709 tablet_service.cc:1511] Processing CreateTablet for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:52.543792 31709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.545882 31545 leader_election.cc:304] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:52.546554 31494 tablet_service.cc:1511] Processing CreateTablet for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:52.547969 31494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.549657 31708 tablet_service.cc:1511] Processing CreateTablet for tablet 0281541aef814bce9263539d80488282 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:52.550686 31772 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:52.551040 31708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0281541aef814bce9263539d80488282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.551353 31772 ts_tablet_manager.cc:1403] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.232s	user 0.012s	sys 0.008s
I20260709 12:16:52.542537 31776 consensus_queue.cc:260] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.554984 31772 raft_consensus.cc:359] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.555759 31772 raft_consensus.cc:385] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.555841 31776 ts_tablet_manager.cc:1434] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.184s	user 0.007s	sys 0.003s
I20260709 12:16:52.556092 31772 raft_consensus.cc:740] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.560990 31813 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.561249 31776 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:52.561547 31813 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.565478 31561 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:52.566847 31561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.568495 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:52.569856 31542 leader_election.cc:343] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.572614 31635 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:52.573894 31813 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.574123 31635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.576330 31560 tablet_service.cc:1511] Processing CreateTablet for tablet abd9c92a3ff84f17a24aeceaffae4ef9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:52.577375 31776 tablet_bootstrap.cc:654] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.578099 31560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd9c92a3ff84f17a24aeceaffae4ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.583065 31579 raft_consensus.cc:2468] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:52.583705 31634 tablet_service.cc:1511] Processing CreateTablet for tablet abd9c92a3ff84f17a24aeceaffae4ef9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:52.584573 31617 leader_election.cc:304] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:52.585130 31634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd9c92a3ff84f17a24aeceaffae4ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.584060 31779 raft_consensus.cc:2749] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.586117 31778 raft_consensus.cc:2804] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:52.586396 31492 tablet_service.cc:1511] Processing CreateTablet for tablet abd9c92a3ff84f17a24aeceaffae4ef9 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:16:52.586957 31778 raft_consensus.cc:697] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:52.587751 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd9c92a3ff84f17a24aeceaffae4ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.588055 31778 consensus_queue.cc:237] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.583361 31493 tablet_service.cc:1511] Processing CreateTablet for tablet 0281541aef814bce9263539d80488282 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:52.592058 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0281541aef814bce9263539d80488282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.542315 31643 tablet_service.cc:1511] Processing CreateTablet for tablet 0281541aef814bce9263539d80488282 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:16:52.598842 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0281541aef814bce9263539d80488282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.556908 31772 consensus_queue.cc:260] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:52.634109 31569 tablet_service.cc:1511] Processing CreateTablet for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:16:52.635602 31569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a8924a0b9c4dbc9ace49508d7b9f7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.638329 31565 tablet_service.cc:1511] Processing CreateTablet for tablet 9da10e70d33e4495babdca0057997818 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:16:52.639006 31772 ts_tablet_manager.cc:1434] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.000s
I20260709 12:16:52.639840 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da10e70d33e4495babdca0057997818. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.644094 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:52.655928 31782 raft_consensus.cc:493] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.661271 31782 raft_consensus.cc:515] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.663532 31782 leader_election.cc:290] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:52.668095 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:52.670240 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:52.670977 31651 raft_consensus.cc:2468] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:52.673449 31808 consensus_queue.cc:1048] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:16:52.680703 31690 leader_election.cc:343] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.680891 31716 tablet_service.cc:1511] Processing CreateTablet for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:16:52.681696 31693 leader_election.cc:304] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:52.682337 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ec83ad93ff4b3abcf4fe9f6005aa7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.683737 31714 tablet_service.cc:1511] Processing CreateTablet for tablet 78d7a21102b64e7f95625a42c84e9ad0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 12:16:52.685652 31616 leader_election.cc:343] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.687486 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:52.688109 31651 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:52.689935 31489 tablet_service.cc:1511] Processing CreateTablet for tablet 78d7a21102b64e7f95625a42c84e9ad0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:52.691248 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d7a21102b64e7f95625a42c84e9ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.693692 31808 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.694200 31808 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.697410 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:52.697989 31772 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:52.698086 31578 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:52.699740 31617 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:52.700675 31778 raft_consensus.cc:2804] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.701066 31778 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.701421 31778 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.683859 31782 raft_consensus.cc:2804] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.708127 31782 raft_consensus.cc:493] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.708492 31782 raft_consensus.cc:3060] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.710049 31808 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.713969 31545 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:52.715176 31779 raft_consensus.cc:2804] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.715546 31779 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.715860 31779 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.722895 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:52.724613 31616 leader_election.cc:343] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.726524 31568 tablet_service.cc:1511] Processing CreateTablet for tablet 78d7a21102b64e7f95625a42c84e9ad0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:16:52.726536 31642 tablet_service.cc:1511] Processing CreateTablet for tablet b76fc75555e64cd3a5762910676fb4e3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:52.728107 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76fc75555e64cd3a5762910676fb4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.728111 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d7a21102b64e7f95625a42c84e9ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.736390 31771 tablet_bootstrap.cc:654] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.753572 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d7a21102b64e7f95625a42c84e9ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.759295 31779 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.770116 31807 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:52.782527 31772 tablet_bootstrap.cc:654] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:52.783723 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:52.792202 31774 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:52.792996 31774 ts_tablet_manager.cc:1403] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.476s	user 0.012s	sys 0.014s
I20260709 12:16:52.785152 31807 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.800287 31779 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.784579 31778 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.818955 31651 raft_consensus.cc:2393] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:52.819161 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
W20260709 12:16:52.822449 31542 leader_election.cc:343] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.822736 31778 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:52.824913 31545 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:52.826385 31562 tablet_service.cc:1511] Processing CreateTablet for tablet 2d3dff40fad349cd90ec48c6ccc15f20 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:52.827276 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
W20260709 12:16:52.829154 31616 leader_election.cc:343] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.829902 31778 consensus_queue.cc:1048] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:52.831157 31807 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
W20260709 12:16:52.831960 31542 consensus_peers.cc:597] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:52.833673 31813 raft_consensus.cc:2749] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:52.835381 31813 consensus_queue.cc:1048] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:52.835819 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:52.836705 31726 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:52.838888 31776 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:52.839458 31776 ts_tablet_manager.cc:1403] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.278s	user 0.042s	sys 0.037s
I20260709 12:16:52.810240 31782 raft_consensus.cc:515] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:52.842077 31467 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:52.843003 31807 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:52.843448 31807 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:52.843947 31807 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.845032 31566 tablet_service.cc:1511] Processing CreateTablet for tablet 68cd2a3bb89a49a6baa7af253a090766 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:52.849800 31776 raft_consensus.cc:359] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.850566 31776 raft_consensus.cc:385] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.829981 31635 tablet_service.cc:1511] Processing CreateTablet for tablet 2d3dff40fad349cd90ec48c6ccc15f20 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:52.850859 31776 raft_consensus.cc:740] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.854687 31776 consensus_queue.cc:260] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.859104 31642 tablet_service.cc:1511] Processing CreateTablet for tablet 68cd2a3bb89a49a6baa7af253a090766 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:52.859485 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:52.860307 31578 raft_consensus.cc:2393] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:52.860571 31642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68cd2a3bb89a49a6baa7af253a090766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.862006 31617 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:52.862736 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:52.862896 31778 raft_consensus.cc:2749] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:52.870740 31468 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.871286 31635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3dff40fad349cd90ec48c6ccc15f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.875959 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:16:52.878130 31690 leader_election.cc:343] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:52.882550 31562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3dff40fad349cd90ec48c6ccc15f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.885215 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:52.886072 31651 raft_consensus.cc:3060] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:52.888769 31566 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68cd2a3bb89a49a6baa7af253a090766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.889328 31782 leader_election.cc:290] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:52.894620 31713 tablet_service.cc:1511] Processing CreateTablet for tablet b76fc75555e64cd3a5762910676fb4e3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:52.896389 31713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76fc75555e64cd3a5762910676fb4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.900974 31488 tablet_service.cc:1511] Processing CreateTablet for tablet b76fc75555e64cd3a5762910676fb4e3 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:16:52.903024 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76fc75555e64cd3a5762910676fb4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.860041 31774 raft_consensus.cc:359] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:52.948210 31774 raft_consensus.cc:385] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:52.960501 31774 raft_consensus.cc:740] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:52.969704 31711 tablet_service.cc:1511] Processing CreateTablet for tablet 2d3dff40fad349cd90ec48c6ccc15f20 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:16:52.970079 31774 consensus_queue.cc:260] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:52.971131 31711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3dff40fad349cd90ec48c6ccc15f20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.972440 31774 ts_tablet_manager.cc:1434] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.179s	user 0.008s	sys 0.002s
W20260709 12:16:52.957538 31387 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 54229.5us
I20260709 12:16:52.968725 31776 ts_tablet_manager.cc:1434] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.129s	user 0.013s	sys 0.025s
I20260709 12:16:52.986668 31776 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:52.984215 31563 tablet_service.cc:1511] Processing CreateTablet for tablet 81a98a7e3a4c46e9a01e39bb9924afc0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:52.990481 31563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a98a7e3a4c46e9a01e39bb9924afc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:52.985064 31489 tablet_service.cc:1511] Processing CreateTablet for tablet 68cd2a3bb89a49a6baa7af253a090766 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:16:52.986896 31710 tablet_service.cc:1511] Processing CreateTablet for tablet 81a98a7e3a4c46e9a01e39bb9924afc0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:52.992592 31807 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:52.993765 31710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a98a7e3a4c46e9a01e39bb9924afc0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:52.963155 31389 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 64656.6us
I20260709 12:16:52.987569 31774 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.006279 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68cd2a3bb89a49a6baa7af253a090766. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.006825 31634 tablet_service.cc:1511] Processing CreateTablet for tablet fe911dcdb0624a2293e42d3d26312b21 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:53.008733 31641 tablet_service.cc:1511] Processing CreateTablet for tablet dbfef93226df40f3a8d07af88af622fb (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:53.010430 31636 tablet_service.cc:1511] Processing CreateTablet for tablet ae5128ee18fc4cedad5f68bd0e804114 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:53.021351 31634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe911dcdb0624a2293e42d3d26312b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.022255 31636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae5128ee18fc4cedad5f68bd0e804114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.023098 31641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbfef93226df40f3a8d07af88af622fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.025223 31569 tablet_service.cc:1511] Processing CreateTablet for tablet ae5128ee18fc4cedad5f68bd0e804114 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:53.019543 31651 raft_consensus.cc:2468] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:53.021868 31568 tablet_service.cc:1511] Processing CreateTablet for tablet dbfef93226df40f3a8d07af88af622fb (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:53.011945 31639 tablet_service.cc:1511] Processing CreateTablet for tablet 544954717af441188c2a2b0ff898f279 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:53.014485 31774 tablet_bootstrap.cc:654] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.020978 31772 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:53.030069 31569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae5128ee18fc4cedad5f68bd0e804114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.030362 31772 ts_tablet_manager.cc:1403] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.333s	user 0.012s	sys 0.016s
I20260709 12:16:53.031090 31492 tablet_service.cc:1511] Processing CreateTablet for tablet 81a98a7e3a4c46e9a01e39bb9924afc0 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:16:53.031581 31832 raft_consensus.cc:493] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.041177 31813 raft_consensus.cc:493] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.070006 31639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544954717af441188c2a2b0ff898f279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.072233 31807 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.072968 31807 raft_consensus.cc:493] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.073493 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:53.073756 31561 tablet_service.cc:1511] Processing CreateTablet for tablet e6c8433c534b4d19bc10a5cb0ca094ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:53.074232 31725 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.075192 31561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c8433c534b4d19bc10a5cb0ca094ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.076280 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81a98a7e3a4c46e9a01e39bb9924afc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.073508 31807 raft_consensus.cc:515] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.077263 31637 tablet_service.cc:1511] Processing CreateTablet for tablet d85a31e6ff114d30b4ae3a4453403a01 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:53.045481 31832 raft_consensus.cc:515] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.078948 31637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85a31e6ff114d30b4ae3a4453403a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.079494 31568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbfef93226df40f3a8d07af88af622fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.084102 31565 tablet_service.cc:1511] Processing CreateTablet for tablet d85a31e6ff114d30b4ae3a4453403a01 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:53.087962 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:53.089828 31693 leader_election.cc:304] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:53.094152 31565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85a31e6ff114d30b4ae3a4453403a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.094503 31831 raft_consensus.cc:2804] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.095252 31831 raft_consensus.cc:697] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:53.096387 31831 consensus_queue.cc:237] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.091091 31806 consensus_queue.cc:1048] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:53.114506 31714 tablet_service.cc:1511] Processing CreateTablet for tablet fe911dcdb0624a2293e42d3d26312b21 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:53.116506 31714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe911dcdb0624a2293e42d3d26312b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.122581 31771 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:53.123360 31771 ts_tablet_manager.cc:1403] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.663s	user 0.012s	sys 0.020s
I20260709 12:16:53.157510 31772 raft_consensus.cc:359] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.158584 31772 raft_consensus.cc:385] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.158918 31772 raft_consensus.cc:740] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.159732 31772 consensus_queue.cc:260] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.160966 31579 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.177270 31771 raft_consensus.cc:359] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.178413 31771 raft_consensus.cc:385] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.178689 31771 raft_consensus.cc:740] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.180218 31813 raft_consensus.cc:515] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.181841 31560 tablet_service.cc:1511] Processing CreateTablet for tablet c130f9abd9da4eaab759d4a5de2c1e86 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:53.183287 31560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c130f9abd9da4eaab759d4a5de2c1e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.185520 31643 tablet_service.cc:1511] Processing CreateTablet for tablet c130f9abd9da4eaab759d4a5de2c1e86 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:53.186249 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.186731 31643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c130f9abd9da4eaab759d4a5de2c1e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.187235 31504 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:53.188789 31772 ts_tablet_manager.cc:1434] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.158s	user 0.005s	sys 0.026s
I20260709 12:16:53.189983 31772 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:53.190599 31567 tablet_service.cc:1511] Processing CreateTablet for tablet 53b9a430b2d54764b10115eae2e0e653 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:53.192025 31567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b9a430b2d54764b10115eae2e0e653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.188679 31542 leader_election.cc:304] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:53.194164 31779 raft_consensus.cc:2804] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.194711 31779 raft_consensus.cc:493] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.195134 31779 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.200199 31832 leader_election.cc:290] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.200436 31640 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6baf18d4c74681ab192824db201eec (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:53.201836 31640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6baf18d4c74681ab192824db201eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.200481 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:53.207772 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:53.209607 31467 leader_election.cc:304] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:53.211483 31779 raft_consensus.cc:515] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.214672 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:53.215298 31504 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.215977 31842 raft_consensus.cc:2749] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:53.179447 31771 consensus_queue.cc:260] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.226944 31504 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:53.228420 31542 leader_election.cc:304] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:53.205339 31776 tablet_bootstrap.cc:654] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.248530 31772 tablet_bootstrap.cc:654] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.250025 31830 raft_consensus.cc:2804] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:53.255502 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:16:53.230098 31693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 140445us
I20260709 12:16:53.257989 31488 tablet_service.cc:1511] Processing CreateTablet for tablet ae5128ee18fc4cedad5f68bd0e804114 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:16:53.258287 31493 tablet_service.cc:1511] Processing CreateTablet for tablet e6c8433c534b4d19bc10a5cb0ca094ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:53.258864 31487 tablet_service.cc:1511] Processing CreateTablet for tablet 544954717af441188c2a2b0ff898f279 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:53.260063 31490 tablet_service.cc:1511] Processing CreateTablet for tablet c130f9abd9da4eaab759d4a5de2c1e86 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:16:53.260576 31708 tablet_service.cc:1511] Processing CreateTablet for tablet dbfef93226df40f3a8d07af88af622fb (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:16:53.261004 31494 tablet_service.cc:1511] Processing CreateTablet for tablet 53b9a430b2d54764b10115eae2e0e653 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:53.261916 31708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbfef93226df40f3a8d07af88af622fb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:53.262420 31616 consensus_peers.cc:597] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:53.208710 31399 catalog_manager.cc:5697] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.263489 31771 ts_tablet_manager.cc:1434] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.140s	user 0.009s	sys 0.001s
I20260709 12:16:53.264576 31771 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:53.264508 31709 tablet_service.cc:1511] Processing CreateTablet for tablet e6c8433c534b4d19bc10a5cb0ca094ba (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:16:53.261608 31716 tablet_service.cc:1511] Processing CreateTablet for tablet d85a31e6ff114d30b4ae3a4453403a01 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:16:53.259351 31488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae5128ee18fc4cedad5f68bd0e804114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.271579 31709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c8433c534b4d19bc10a5cb0ca094ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.263530 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.275962 31651 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:53.277133 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:53.278266 31470 leader_election.cc:304] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:53.279116 31832 raft_consensus.cc:2764] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:16:53.279803 31715 tablet_service.cc:1511] Processing CreateTablet for tablet 544954717af441188c2a2b0ff898f279 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:16:53.280052 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:53.282105 31492 tablet_service.cc:1511] Processing CreateTablet for tablet fe911dcdb0624a2293e42d3d26312b21 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:16:53.259546 31807 leader_election.cc:290] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:53.288553 31715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544954717af441188c2a2b0ff898f279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.292532 31830 raft_consensus.cc:697] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:53.283463 31779 leader_election.cc:290] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:53.300998 31579 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:53.265810 31717 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6baf18d4c74681ab192824db201eec (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:53.305704 31494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b9a430b2d54764b10115eae2e0e653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.264462 31712 tablet_service.cc:1511] Processing CreateTablet for tablet 53b9a430b2d54764b10115eae2e0e653 (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:16:53.291124 31651 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.319669 31712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53b9a430b2d54764b10115eae2e0e653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.320583 31504 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.330686 31489 tablet_service.cc:1511] Processing CreateTablet for tablet 1f6baf18d4c74681ab192824db201eec (DEFAULT_TABLE table=test-workload [id=b11ddd3742ea4c419adf9732709e25d0]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:16:53.332130 31489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6baf18d4c74681ab192824db201eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.338276 31725 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:53.339788 31467 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:53.340764 31807 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.341012 31831 raft_consensus.cc:493] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.341380 31807 raft_consensus.cc:697] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:53.341549 31831 raft_consensus.cc:515] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.342465 31807 consensus_queue.cc:237] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.340826 31813 leader_election.cc:290] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:53.363056 31716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85a31e6ff114d30b4ae3a4453403a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.295111 31830 consensus_queue.cc:237] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.329339 31651 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:53.367206 31490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c130f9abd9da4eaab759d4a5de2c1e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.385995 31776 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.386731 31776 ts_tablet_manager.cc:1403] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.400s	user 0.012s	sys 0.011s
I20260709 12:16:53.392086 31771 tablet_bootstrap.cc:654] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.397581 31717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f6baf18d4c74681ab192824db201eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.407181 31831 leader_election.cc:290] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:53.409171 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.409629 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.409945 31651 raft_consensus.cc:2468] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:53.410341 31503 raft_consensus.cc:2468] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:53.411142 31493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6c8433c534b4d19bc10a5cb0ca094ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.378340 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.417377 31652 raft_consensus.cc:2376] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1. Re-sending same reply.
I20260709 12:16:53.423282 31492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe911dcdb0624a2293e42d3d26312b21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.427803 31690 leader_election.cc:304] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:53.428809 31831 raft_consensus.cc:2804] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.429188 31831 raft_consensus.cc:493] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.429590 31831 raft_consensus.cc:3060] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.457700 31774 tablet_bootstrap.cc:492] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.452304 31487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 544954717af441188c2a2b0ff898f279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.473054 31772 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:53.473658 31772 ts_tablet_manager.cc:1403] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.284s	user 0.032s	sys 0.033s
I20260709 12:16:53.477593 31774 ts_tablet_manager.cc:1403] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.504s	user 0.017s	sys 0.012s
I20260709 12:16:53.495126 31402 catalog_manager.cc:5697] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.463233 31831 raft_consensus.cc:515] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.506345 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:53.507097 31652 raft_consensus.cc:3060] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.510118 31831 leader_election.cc:290] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:53.509960 31776 raft_consensus.cc:359] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.510852 31776 raft_consensus.cc:385] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.512651 31772 raft_consensus.cc:359] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.513391 31772 raft_consensus.cc:385] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.513720 31772 raft_consensus.cc:740] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.514472 31772 consensus_queue.cc:260] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.515195 31774 raft_consensus.cc:359] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.515872 31652 raft_consensus.cc:2468] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:53.516984 31772 ts_tablet_manager.cc:1434] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:16:53.517217 31693 leader_election.cc:304] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:53.518152 31772 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:53.511179 31776 raft_consensus.cc:740] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.515957 31774 raft_consensus.cc:385] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:53.519263 31848 log.cc:927] Time spent T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:16:53.520074 31774 raft_consensus.cc:740] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.520324 31776 consensus_queue.cc:260] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.520970 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af22442d6a9141f2a4607ba57cf4e0ba" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:53.521658 31504 raft_consensus.cc:3060] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.521082 31774 consensus_queue.cc:260] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.523121 31776 ts_tablet_manager.cc:1434] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.136s	user 0.009s	sys 0.000s
I20260709 12:16:53.524554 31776 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.526934 31774 ts_tablet_manager.cc:1434] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.001s
I20260709 12:16:53.528064 31774 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.529923 31504 raft_consensus.cc:2468] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:53.519650 31831 raft_consensus.cc:2804] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.531481 31831 raft_consensus.cc:697] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:53.531693 31776 tablet_bootstrap.cc:654] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.532382 31831 consensus_queue.cc:237] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.547415 31399 catalog_manager.cc:5697] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.549724 31771 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:53.550292 31771 ts_tablet_manager.cc:1403] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.286s	user 0.011s	sys 0.008s
I20260709 12:16:53.564823 31774 tablet_bootstrap.cc:654] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.567052 31771 raft_consensus.cc:359] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.567905 31771 raft_consensus.cc:385] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.568200 31771 raft_consensus.cc:740] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.570490 31772 tablet_bootstrap.cc:654] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.575208 31776 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.572526 31402 catalog_manager.cc:5697] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.576107 31776 ts_tablet_manager.cc:1403] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.052s	user 0.030s	sys 0.020s
I20260709 12:16:53.579591 31771 consensus_queue.cc:260] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.580613 31776 raft_consensus.cc:359] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.581372 31776 raft_consensus.cc:385] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.581709 31776 raft_consensus.cc:740] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.582628 31771 ts_tablet_manager.cc:1434] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 12:16:53.582593 31776 consensus_queue.cc:260] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.583604 31774 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.583873 31771 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:53.584245 31774 ts_tablet_manager.cc:1403] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.006s
I20260709 12:16:53.585615 31776 ts_tablet_manager.cc:1434] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:16:53.588320 31774 raft_consensus.cc:359] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.589061 31772 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:53.589154 31774 raft_consensus.cc:385] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.589648 31772 ts_tablet_manager.cc:1403] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.006s
I20260709 12:16:53.590842 31771 tablet_bootstrap.cc:654] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.592617 31772 raft_consensus.cc:359] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.593295 31772 raft_consensus.cc:385] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.596403 31399 catalog_manager.cc:5697] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:53.589756 31774 raft_consensus.cc:740] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.601359 31774 consensus_queue.cc:260] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.604044 31774 ts_tablet_manager.cc:1434] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:16:53.593629 31772 raft_consensus.cc:740] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.605197 31774 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.605278 31772 consensus_queue.cc:260] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.607806 31807 raft_consensus.cc:493] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.608510 31807 raft_consensus.cc:515] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.610947 31776 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.612416 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.613202 31774 tablet_bootstrap.cc:654] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.613545 31652 raft_consensus.cc:2393] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:53.619185 31776 tablet_bootstrap.cc:654] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.620339 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd72db62f85b4234aee25b5eda5abddd" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:53.622603 31467 leader_election.cc:304] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:53.623554 31832 raft_consensus.cc:3060] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.624449 31771 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:53.624960 31771 ts_tablet_manager.cc:1403] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.041s	user 0.020s	sys 0.018s
I20260709 12:16:53.627915 31771 raft_consensus.cc:359] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.628593 31771 raft_consensus.cc:385] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.628850 31771 raft_consensus.cc:740] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.629724 31771 consensus_queue.cc:260] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.630484 31807 leader_election.cc:290] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:53.634411 31771 ts_tablet_manager.cc:1434] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:16:53.635396 31771 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:53.631665 31776 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.640851 31776 ts_tablet_manager.cc:1403] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.003s
I20260709 12:16:53.641652 31771 tablet_bootstrap.cc:654] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.644523 31776 raft_consensus.cc:359] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.646230 31776 raft_consensus.cc:385] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.646535 31776 raft_consensus.cc:740] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.647228 31776 consensus_queue.cc:260] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.648540 31772 ts_tablet_manager.cc:1434] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 12:16:53.649350 31776 ts_tablet_manager.cc:1434] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:16:53.650028 31772 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:53.654681 31776 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.661351 31776 tablet_bootstrap.cc:654] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.661900 31772 tablet_bootstrap.cc:654] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.669651 31808 raft_consensus.cc:493] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.670182 31808 raft_consensus.cc:515] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.672385 31808 leader_election.cc:290] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.674047 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:53.675014 31578 raft_consensus.cc:2468] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:53.676362 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.677258 31503 raft_consensus.cc:2468] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:53.678995 31616 leader_election.cc:304] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:53.680223 31808 raft_consensus.cc:2804] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.680724 31808 raft_consensus.cc:493] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.681170 31808 raft_consensus.cc:3060] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.683827 31832 raft_consensus.cc:2749] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:53.687140 31771 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:53.687687 31771 ts_tablet_manager.cc:1403] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.053s	user 0.006s	sys 0.006s
I20260709 12:16:53.689724 31831 consensus_queue.cc:1048] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:53.690963 31771 raft_consensus.cc:359] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.691912 31771 raft_consensus.cc:385] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.692288 31771 raft_consensus.cc:740] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.692567 31774 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.693100 31774 ts_tablet_manager.cc:1403] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.088s	user 0.051s	sys 0.024s
I20260709 12:16:53.693471 31771 consensus_queue.cc:260] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.697536 31774 raft_consensus.cc:359] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.698258 31774 raft_consensus.cc:385] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.698534 31774 raft_consensus.cc:740] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.699249 31774 consensus_queue.cc:260] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.701951 31774 ts_tablet_manager.cc:1434] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:16:53.703838 31808 raft_consensus.cc:515] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.710090 31503 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.716153 31774 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.716884 31771 ts_tablet_manager.cc:1434] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:16:53.720618 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:53.724032 31774 tablet_bootstrap.cc:654] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.724117 31776 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.724479 31502 raft_consensus.cc:3060] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.724759 31776 ts_tablet_manager.cc:1403] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.070s	user 0.031s	sys 0.022s
I20260709 12:16:53.727708 31776 raft_consensus.cc:359] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.728355 31776 raft_consensus.cc:385] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.728628 31776 raft_consensus.cc:740] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.723093 31772 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:53.729420 31776 consensus_queue.cc:260] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.730180 31772 ts_tablet_manager.cc:1403] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.008s
I20260709 12:16:53.730697 31782 consensus_queue.cc:1048] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:53.734318 31772 raft_consensus.cc:359] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.742632 31776 ts_tablet_manager.cc:1434] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:16:53.745308 31776 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.746326 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c506263e3b0d4ce98f8336aec8159188" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:53.745947 31771 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:53.747063 31578 raft_consensus.cc:3060] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.746179 31772 raft_consensus.cc:385] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.747671 31772 raft_consensus.cc:740] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.748992 31502 raft_consensus.cc:2468] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:53.750603 31616 leader_election.cc:304] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:53.751595 31778 raft_consensus.cc:2804] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.752225 31778 raft_consensus.cc:697] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:53.754087 31778 consensus_queue.cc:237] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.759774 31774 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.760331 31774 ts_tablet_manager.cc:1403] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.012s
I20260709 12:16:53.763083 31774 raft_consensus.cc:359] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.763738 31774 raft_consensus.cc:385] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.764010 31774 raft_consensus.cc:740] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.755535 31772 consensus_queue.cc:260] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.764688 31774 consensus_queue.cc:260] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.768893 31772 ts_tablet_manager.cc:1434] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:16:53.770054 31772 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:53.774040 31776 tablet_bootstrap.cc:654] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.770794 31774 ts_tablet_manager.cc:1434] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:16:53.743835 31808 leader_election.cc:290] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.777974 31774 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.782372 31776 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.782915 31776 ts_tablet_manager.cc:1403] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.000s
I20260709 12:16:53.785418 31806 raft_consensus.cc:493] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.785969 31806 raft_consensus.cc:515] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.788049 31774 tablet_bootstrap.cc:654] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.789783 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:53.790200 31772 tablet_bootstrap.cc:654] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.790549 31579 raft_consensus.cc:2468] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:53.792861 31617 leader_election.cc:304] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:53.793411 31776 raft_consensus.cc:359] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.794212 31776 raft_consensus.cc:385] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.794502 31776 raft_consensus.cc:740] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.795267 31776 consensus_queue.cc:260] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.797206 31776 ts_tablet_manager.cc:1434] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 12:16:53.799985 31578 raft_consensus.cc:2468] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:53.803129 31808 raft_consensus.cc:2804] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.803690 31808 raft_consensus.cc:493] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.804144 31808 raft_consensus.cc:3060] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.791194 31806 leader_election.cc:290] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.813555 31774 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.813655 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.814162 31774 ts_tablet_manager.cc:1403] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.003s
I20260709 12:16:53.814221 31771 tablet_bootstrap.cc:654] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:53.815570 31616 leader_election.cc:343] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.815575 31832 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.817397 31774 raft_consensus.cc:359] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.819262 31774 raft_consensus.cc:385] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.819666 31774 raft_consensus.cc:740] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.820500 31774 consensus_queue.cc:260] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.821583 31776 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.822681 31774 ts_tablet_manager.cc:1434] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:53.823623 31774 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.817567 31832 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.826084 31832 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.827880 31776 tablet_bootstrap.cc:654] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.829341 31774 tablet_bootstrap.cc:654] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.833782 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:53.834748 31725 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:53.838485 31467 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:53.840173 31782 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.840703 31782 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.841849 31400 catalog_manager.cc:5697] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.843715 31807 raft_consensus.cc:2804] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.844116 31807 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.844429 31807 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.844822 31808 raft_consensus.cc:515] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.845877 31832 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.846544 31832 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.847723 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:53.847944 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.848841 31832 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:53.842804 31782 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:53.839920 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:53.849490 31693 leader_election.cc:343] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:53.850639 31468 leader_election.cc:343] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.853546 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:53.854200 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.854846 31652 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:53.856621 31807 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
W20260709 12:16:53.860124 31616 leader_election.cc:343] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.861202 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:53.862097 31725 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.862038 31470 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:53.854228 31726 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:53.867210 31807 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.868099 31807 raft_consensus.cc:2804] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.868506 31807 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.868863 31807 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.867236 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.873086 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
W20260709 12:16:53.873116 31690 leader_election.cc:343] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.875041 31690 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:53.876137 31782 raft_consensus.cc:2749] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:53.876492 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:53.877182 31578 raft_consensus.cc:3060] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:53.877049 31468 leader_election.cc:343] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:53.863835 31832 consensus_queue.cc:1048] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:16:53.884797 31808 leader_election.cc:290] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.891846 31771 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:53.892408 31771 ts_tablet_manager.cc:1403] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.147s	user 0.012s	sys 0.001s
I20260709 12:16:53.893868 31578 raft_consensus.cc:2468] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:53.895212 31771 raft_consensus.cc:359] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.895918 31771 raft_consensus.cc:385] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.895768 31617 leader_election.cc:304] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:53.896379 31771 raft_consensus.cc:740] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.897523 31808 raft_consensus.cc:2804] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.898525 31808 raft_consensus.cc:697] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:53.899587 31808 consensus_queue.cc:237] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.897341 31771 consensus_queue.cc:260] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.907202 31771 ts_tablet_manager.cc:1434] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:16:53.907711 31772 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:53.908267 31771 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:53.908422 31772 ts_tablet_manager.cc:1403] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.139s	user 0.022s	sys 0.035s
W20260709 12:16:53.909965 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 48062.4us
I20260709 12:16:53.915679 31772 raft_consensus.cc:359] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.917285 31772 raft_consensus.cc:385] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.917613 31772 raft_consensus.cc:740] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.918318 31772 consensus_queue.cc:260] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.919483 31725 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:53.919793 31772 ts_tablet_manager.cc:1434] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:53.921223 31467 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:53.922456 31877 raft_consensus.cc:2804] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.923745 31877 raft_consensus.cc:697] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:53.924778 31774 tablet_bootstrap.cc:492] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:53.925287 31774 ts_tablet_manager.cc:1403] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.102s	user 0.008s	sys 0.007s
I20260709 12:16:53.925899 31846 raft_consensus.cc:493] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.926405 31846 raft_consensus.cc:515] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.928750 31846 leader_election.cc:290] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:53.924659 31877 consensus_queue.cc:237] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.930663 31774 raft_consensus.cc:359] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.931663 31774 raft_consensus.cc:385] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.931938 31774 raft_consensus.cc:740] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.932730 31774 consensus_queue.cc:260] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:53.934825 31774 ts_tablet_manager.cc:1434] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:53.935276 31772 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:53.936051 31776 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:53.936627 31776 ts_tablet_manager.cc:1403] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.115s	user 0.025s	sys 0.010s
I20260709 12:16:53.929574 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b15073c4e54ef1af602f9c83702642" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:53.938222 31652 raft_consensus.cc:2468] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:53.939507 31807 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:53.940275 31545 leader_election.cc:304] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:53.941743 31807 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:53.937295 31774 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:53.943095 31402 catalog_manager.cc:5697] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:53.944356 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:53.944917 31652 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.946635 31846 raft_consensus.cc:2804] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.946790 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:53.947623 31727 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.953200 31776 raft_consensus.cc:359] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.955150 31776 raft_consensus.cc:385] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.955425 31776 raft_consensus.cc:740] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.955817 31652 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:53.947105 31846 raft_consensus.cc:493] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.956161 31776 consensus_queue.cc:260] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:53.956633 31782 raft_consensus.cc:493] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:53.961369 31782 raft_consensus.cc:515] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:53.963505 31782 leader_election.cc:290] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:53.953403 31772 tablet_bootstrap.cc:654] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.964519 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:53.949962 31774 tablet_bootstrap.cc:654] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.965183 31503 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:53.965234 31832 consensus_queue.cc:1048] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:53.966579 31690 leader_election.cc:304] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:53.960281 31776 ts_tablet_manager.cc:1434] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:16:53.968487 31776 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:53.967415 31782 raft_consensus.cc:2804] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:53.971484 31782 raft_consensus.cc:493] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:53.971872 31782 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.970715 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:53.976922 31578 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:53.967237 31470 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:53.978271 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b15073c4e54ef1af602f9c83702642" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:53.978910 31725 raft_consensus.cc:2468] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:53.983489 31771 tablet_bootstrap.cc:654] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.967547 31846 raft_consensus.cc:3060] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:53.998355 31877 raft_consensus.cc:2804] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:53.999413 31727 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:54.000874 31831 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.002442 31831 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.001937 31877 raft_consensus.cc:697] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:54.004086 31877 consensus_queue.cc:237] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.004680 31831 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.006062 31776 tablet_bootstrap.cc:654] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.016575 31772 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:54.011099 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.017367 31772 ts_tablet_manager.cc:1403] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.086s	user 0.023s	sys 0.017s
I20260709 12:16:54.027328 31772 raft_consensus.cc:359] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.027729 31578 raft_consensus.cc:2393] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:54.028455 31772 raft_consensus.cc:385] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.028884 31772 raft_consensus.cc:740] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.022338 31402 catalog_manager.cc:5697] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.029901 31772 consensus_queue.cc:260] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.035106 31772 ts_tablet_manager.cc:1434] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:54.036171 31772 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.042822 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.044049 31652 raft_consensus.cc:2393] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:54.046182 31693 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:54.047435 31831 consensus_queue.cc:1048] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.049012 31831 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.011240 31782 raft_consensus.cc:515] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.056021 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:54.056737 31772 tablet_bootstrap.cc:654] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.056854 31503 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.071939 31846 raft_consensus.cc:515] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.077705 31782 leader_election.cc:290] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:54.088961 31776 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:54.089056 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:54.089766 31776 ts_tablet_manager.cc:1403] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.122s	user 0.011s	sys 0.003s
I20260709 12:16:54.089880 31578 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:54.090691 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 132479us
I20260709 12:16:54.092672 31776 raft_consensus.cc:359] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.096360 31503 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.093389 31776 raft_consensus.cc:385] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.098999 31776 raft_consensus.cc:740] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.098062 31813 consensus_queue.cc:1048] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:54.097925 31690 leader_election.cc:304] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.093348 31831 raft_consensus.cc:2749] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.109127 31772 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:54.104902 31776 consensus_queue.cc:260] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.106524 31399 catalog_manager.cc:5697] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.111831 31772 ts_tablet_manager.cc:1403] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.076s	user 0.020s	sys 0.003s
I20260709 12:16:54.118956 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b15073c4e54ef1af602f9c83702642" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.119745 31651 raft_consensus.cc:3060] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.121250 31831 raft_consensus.cc:2804] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.140574 31776 ts_tablet_manager.cc:1434] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.001s
I20260709 12:16:54.135987 31831 raft_consensus.cc:697] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:54.146575 31831 consensus_queue.cc:237] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.152982 31846 leader_election.cc:290] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.153789 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75b15073c4e54ef1af602f9c83702642" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:54.155539 31771 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:54.157250 31771 ts_tablet_manager.cc:1403] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.249s	user 0.020s	sys 0.006s
I20260709 12:16:54.156322 31776 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:54.157771 31813 consensus_queue.cc:1048] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:54.160970 31727 raft_consensus.cc:3060] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.172098 31772 raft_consensus.cc:359] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.173077 31772 raft_consensus.cc:385] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.173447 31772 raft_consensus.cc:740] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.184880 31578 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.187891 31772 consensus_queue.cc:260] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.193975 31651 raft_consensus.cc:2468] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.195477 31545 leader_election.cc:304] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.197005 31808 consensus_queue.cc:1048] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.197639 31846 raft_consensus.cc:2804] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.198325 31846 raft_consensus.cc:697] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:54.199141 31774 tablet_bootstrap.cc:492] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:54.199630 31774 ts_tablet_manager.cc:1403] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.263s	user 0.028s	sys 0.044s
I20260709 12:16:54.199422 31846 consensus_queue.cc:237] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.202363 31774 raft_consensus.cc:359] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.203084 31774 raft_consensus.cc:385] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.204265 31774 raft_consensus.cc:740] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.204977 31774 consensus_queue.cc:260] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.207706 31774 ts_tablet_manager.cc:1434] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:54.213346 31772 ts_tablet_manager.cc:1434] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.001s
I20260709 12:16:54.213591 31771 raft_consensus.cc:359] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.214490 31771 raft_consensus.cc:385] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.212030 31776 tablet_bootstrap.cc:654] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.214864 31771 raft_consensus.cc:740] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.217419 31771 consensus_queue.cc:260] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.218607 31772 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.213591 31774 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:54.229039 31771 ts_tablet_manager.cc:1434] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.071s	user 0.004s	sys 0.004s
I20260709 12:16:54.243551 31846 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.244119 31846 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.247465 31831 consensus_queue.cc:1048] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.249712 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:54.250557 31725 raft_consensus.cc:2393] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:54.236683 31727 raft_consensus.cc:2468] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.261612 31771 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:54.265034 31772 tablet_bootstrap.cc:654] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.270576 31873 consensus_queue.cc:1048] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.275246 31774 tablet_bootstrap.cc:654] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.279366 31399 catalog_manager.cc:5697] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.306329 31884 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.306958 31884 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.307790 31846 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
W20260709 12:16:54.302258 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:54.309278 31884 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.335125 31846 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.335012 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:54.339951 31846 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.340683 31727 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:54.334995 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.342374 31467 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.342532 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
W20260709 12:16:54.343120 31470 leader_election.cc:343] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.343971 31884 raft_consensus.cc:2804] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.344385 31884 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.344718 31884 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.346509 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.347383 31651 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.344760 31846 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.348973 31545 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.352392 31542 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:54.352980 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:54.353861 31727 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.355861 31846 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.357878 31906 raft_consensus.cc:2804] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:54.358278 31906 raft_consensus.cc:493] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.358599 31906 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.361660 31897 raft_consensus.cc:493] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.362201 31897 raft_consensus.cc:515] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.364488 31897 leader_election.cc:290] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.365810 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bd4922c2764afd8737751cb3d17cb8" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.366606 31579 raft_consensus.cc:2468] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:54.365756 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bd4922c2764afd8737751cb3d17cb8" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.368814 31691 leader_election.cc:304] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.369005 31651 raft_consensus.cc:2468] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:54.370245 31897 raft_consensus.cc:2804] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.370617 31897 raft_consensus.cc:493] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.370949 31897 raft_consensus.cc:3060] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.374923 31846 raft_consensus.cc:2749] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:54.375495 31898 log.cc:927] Time spent T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:16:54.376237 31877 consensus_queue.cc:1048] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.375641 31776 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:54.378698 31776 ts_tablet_manager.cc:1403] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.223s	user 0.011s	sys 0.007s
I20260709 12:16:54.381863 31776 raft_consensus.cc:359] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.382551 31776 raft_consensus.cc:385] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.382833 31776 raft_consensus.cc:740] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.383697 31776 consensus_queue.cc:260] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.386027 31776 ts_tablet_manager.cc:1434] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:54.390727 31776 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:54.396911 31771 tablet_bootstrap.cc:654] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.399353 31807 raft_consensus.cc:493] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.399999 31807 raft_consensus.cc:515] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.400743 31806 consensus_queue.cc:1048] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.404511 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.425722 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33acdf64b87145ab8a38a66147ae23b5" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.412492 31807 leader_election.cc:290] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.428763 31807 consensus_queue.cc:1048] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.430454 31906 raft_consensus.cc:515] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.445082 31906 leader_election.cc:290] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.448235 31897 raft_consensus.cc:515] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.441198 31776 tablet_bootstrap.cc:654] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.457511 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.458225 31651 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.459734 31772 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:54.460418 31772 ts_tablet_manager.cc:1403] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.242s	user 0.048s	sys 0.027s
I20260709 12:16:54.463044 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bd4922c2764afd8737751cb3d17cb8" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.463713 31652 raft_consensus.cc:3060] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.463464 31772 raft_consensus.cc:359] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.464406 31772 raft_consensus.cc:385] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.464732 31772 raft_consensus.cc:740] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.465585 31772 consensus_queue.cc:260] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.457762 31897 leader_election.cc:290] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.472612 31772 ts_tablet_manager.cc:1434] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:16:54.473873 31772 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.457551 31468 leader_election.cc:304] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
W20260709 12:16:54.476588 31468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 48166us
I20260709 12:16:54.477517 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "74bd4922c2764afd8737751cb3d17cb8" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:54.478137 31579 raft_consensus.cc:3060] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.480743 31772 tablet_bootstrap.cc:654] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.467146 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0f4df696784a248aaa0b98db69dfa0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:54.481942 31725 raft_consensus.cc:3060] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.488106 31774 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:54.488634 31774 ts_tablet_manager.cc:1403] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.275s	user 0.023s	sys 0.022s
I20260709 12:16:54.487969 31401 catalog_manager.cc:5697] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.489583 31807 raft_consensus.cc:2749] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.492488 31651 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:16:54.493768 31545 leader_election.cc:304] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.494575 31906 raft_consensus.cc:2804] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:54.495049 31906 raft_consensus.cc:697] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:54.496013 31906 consensus_queue.cc:237] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.500619 31774 raft_consensus.cc:359] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.502789 31774 raft_consensus.cc:385] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.503192 31774 raft_consensus.cc:740] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.507236 31774 consensus_queue.cc:260] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.452822 31884 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.513914 31774 ts_tablet_manager.cc:1434] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.002s
I20260709 12:16:54.514808 31774 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:54.542747 31771 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:54.543422 31771 ts_tablet_manager.cc:1403] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.282s	user 0.019s	sys 0.026s
I20260709 12:16:54.537318 31884 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.539561 31652 raft_consensus.cc:2468] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.548902 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.535954 31725 raft_consensus.cc:2468] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:16:54.549903 31693 leader_election.cc:304] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.547816 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:54.551007 31897 raft_consensus.cc:2804] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.551116 31727 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.551575 31897 raft_consensus.cc:697] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:54.552453 31897 consensus_queue.cc:237] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.533361 31807 consensus_queue.cc:1048] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:16:54.553893 31579 raft_consensus.cc:2468] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.555668 31503 raft_consensus.cc:3060] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.573266 31727 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:54.574771 31467 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.575825 31884 raft_consensus.cc:2804] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.576363 31884 raft_consensus.cc:697] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:54.577155 31884 consensus_queue.cc:237] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.578490 31808 consensus_queue.cc:1048] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.581591 31772 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:54.582120 31772 ts_tablet_manager.cc:1403] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.108s	user 0.018s	sys 0.022s
I20260709 12:16:54.587222 31774 tablet_bootstrap.cc:654] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.584153 31877 consensus_queue.cc:1048] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:54.610446 31772 raft_consensus.cc:359] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.620635 31772 raft_consensus.cc:385] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.622840 31772 raft_consensus.cc:740] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.625808 31772 consensus_queue.cc:260] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.627259 31776 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:54.627782 31776 ts_tablet_manager.cc:1403] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.237s	user 0.022s	sys 0.003s
I20260709 12:16:54.628259 31772 ts_tablet_manager.cc:1434] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 12:16:54.629259 31772 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.630597 31776 raft_consensus.cc:359] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.631285 31776 raft_consensus.cc:385] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.631572 31776 raft_consensus.cc:740] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.633718 31771 raft_consensus.cc:359] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.634272 31771 raft_consensus.cc:385] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.634472 31771 raft_consensus.cc:740] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.635670 31776 consensus_queue.cc:260] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.638092 31776 ts_tablet_manager.cc:1434] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:16:54.639094 31776 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:16:54.639372 31470 leader_election.cc:343] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.644017 31771 consensus_queue.cc:260] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.648047 31771 ts_tablet_manager.cc:1434] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.104s	user 0.005s	sys 0.000s
I20260709 12:16:54.649138 31906 consensus_queue.cc:1048] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.655714 31399 catalog_manager.cc:5697] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.662647 31771 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:54.683597 31776 tablet_bootstrap.cc:654] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.692167 31774 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:54.692732 31774 ts_tablet_manager.cc:1403] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.178s	user 0.009s	sys 0.024s
I20260709 12:16:54.691053 31400 catalog_manager.cc:5697] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 2, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 2 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.694008 31897 consensus_queue.cc:1048] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.695525 31774 raft_consensus.cc:359] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.696204 31774 raft_consensus.cc:385] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.696286 31772 tablet_bootstrap.cc:654] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.696521 31774 raft_consensus.cc:740] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.702970 31771 tablet_bootstrap.cc:654] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.697300 31774 consensus_queue.cc:260] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.707068 31774 ts_tablet_manager.cc:1434] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:54.708024 31774 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:54.712754 31893 consensus_queue.cc:1048] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.714929 31774 tablet_bootstrap.cc:654] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.715593 31813 consensus_queue.cc:1048] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.718814 31401 catalog_manager.cc:5697] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.732374 31774 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:54.732895 31774 ts_tablet_manager.cc:1403] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 12:16:54.744596 31774 raft_consensus.cc:359] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.745330 31774 raft_consensus.cc:385] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.747615 31774 raft_consensus.cc:740] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.748268 31774 consensus_queue.cc:260] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.752465 31774 ts_tablet_manager.cc:1434] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.003s
I20260709 12:16:54.753530 31774 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:54.759299 31906 raft_consensus.cc:493] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.759876 31906 raft_consensus.cc:515] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.760993 31897 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.761993 31906 leader_election.cc:290] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:54.761705 31897 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.764138 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecff280f5652462db2e6473ea7424a2f" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:54.764879 31725 raft_consensus.cc:2468] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:54.765576 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.766425 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecff280f5652462db2e6473ea7424a2f" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:54.767052 31502 raft_consensus.cc:2468] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
W20260709 12:16:54.767571 31693 leader_election.cc:343] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.771663 31542 leader_election.cc:304] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.772728 31813 raft_consensus.cc:2804] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.773093 31813 raft_consensus.cc:493] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.773218 31772 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:54.773465 31813 raft_consensus.cc:3060] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.773787 31772 ts_tablet_manager.cc:1403] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.000s
I20260709 12:16:54.776055 31897 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:54.776623 31772 raft_consensus.cc:359] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.777333 31772 raft_consensus.cc:385] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.782780 31813 raft_consensus.cc:515] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.785734 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecff280f5652462db2e6473ea7424a2f" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:54.786366 31725 raft_consensus.cc:3060] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.779541 31772 raft_consensus.cc:740] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.788259 31774 tablet_bootstrap.cc:654] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.788435 31772 consensus_queue.cc:260] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.791558 31772 ts_tablet_manager.cc:1434] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:16:54.793463 31772 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.798565 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:54.793829 31782 raft_consensus.cc:493] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.807128 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecff280f5652462db2e6473ea7424a2f" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:54.806972 31782 raft_consensus.cc:515] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.807776 31501 raft_consensus.cc:3060] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.810105 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.809290 31782 leader_election.cc:290] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.812283 31577 raft_consensus.cc:2468] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:54.800580 31813 leader_election.cc:290] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:54.814226 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.815991 31691 leader_election.cc:304] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:54.817339 31501 raft_consensus.cc:2468] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.818996 31542 leader_election.cc:304] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.820034 31813 raft_consensus.cc:2804] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.820546 31813 raft_consensus.cc:697] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:54.820741 31725 raft_consensus.cc:2468] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:54.821636 31897 raft_consensus.cc:2804] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.822118 31897 raft_consensus.cc:493] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:16:54.828595 31693 leader_election.cc:343] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.829129 31897 raft_consensus.cc:3060] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.821650 31813 consensus_queue.cc:237] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.837996 31772 tablet_bootstrap.cc:654] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.838586 31897 raft_consensus.cc:515] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.854967 31897 leader_election.cc:290] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.854806 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:54.855597 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.855749 31577 raft_consensus.cc:3060] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:54.857085 31693 leader_election.cc:343] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.857846 31771 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:54.858433 31771 ts_tablet_manager.cc:1403] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.196s	user 0.041s	sys 0.016s
I20260709 12:16:54.858314 31884 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.859104 31884 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.860145 31776 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:54.860800 31776 ts_tablet_manager.cc:1403] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.222s	user 0.021s	sys 0.019s
I20260709 12:16:54.861994 31906 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.862514 31906 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.862972 31771 raft_consensus.cc:359] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.865976 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.866003 31771 raft_consensus.cc:385] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.866839 31771 raft_consensus.cc:740] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.864679 31776 raft_consensus.cc:359] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.867997 31884 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.867797 31776 raft_consensus.cc:385] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.868726 31776 raft_consensus.cc:740] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.869544 31776 consensus_queue.cc:260] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:54.870000 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.872303 31579 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:54.867806 31771 consensus_queue.cc:260] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.864719 31906 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.874660 31776 ts_tablet_manager.cc:1434] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.003s
I20260709 12:16:54.876246 31776 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:54.877832 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.874033 31468 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.878933 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:54.879354 31884 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:54.876575 31690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 72994.8us
I20260709 12:16:54.879776 31884 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.880124 31884 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.882097 31772 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
W20260709 12:16:54.882448 31690 leader_election.cc:343] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.882611 31772 ts_tablet_manager.cc:1403] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.089s	user 0.023s	sys 0.013s
I20260709 12:16:54.882956 31690 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:54.886102 31772 raft_consensus.cc:359] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.886608 31774 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:54.886806 31772 raft_consensus.cc:385] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.887173 31774 ts_tablet_manager.cc:1403] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.001s
I20260709 12:16:54.887157 31772 raft_consensus.cc:740] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.878463 31651 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:54.888196 31772 consensus_queue.cc:260] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.886976 31831 raft_consensus.cc:2749] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.890034 31545 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:54.891428 31906 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.891788 31906 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.892124 31906 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.892706 31774 raft_consensus.cc:359] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.893390 31774 raft_consensus.cc:385] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.893669 31774 raft_consensus.cc:740] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.894322 31774 consensus_queue.cc:260] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.896139 31577 raft_consensus.cc:2468] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.897498 31691 leader_election.cc:304] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:54.898488 31776 tablet_bootstrap.cc:654] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.902268 31771 ts_tablet_manager.cc:1434] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.001s
I20260709 12:16:54.905963 31651 raft_consensus.cc:1275] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:54.907649 31846 consensus_queue.cc:1048] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.910492 31771 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:54.914469 31906 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.915304 31893 raft_consensus.cc:2804] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.915846 31893 raft_consensus.cc:697] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:54.916947 31772 ts_tablet_manager.cc:1434] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:16:54.916684 31893 consensus_queue.cc:237] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:54.919956 31772 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:54.914553 31897 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.920545 31897 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.922628 31897 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:54.923521 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:54.926129 31774 ts_tablet_manager.cc:1434] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:16:54.926241 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:54.927184 31774 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:54.928918 31877 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.929517 31877 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
W20260709 12:16:54.927635 31690 leader_election.cc:343] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.923178 31771 tablet_bootstrap.cc:654] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.934691 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:54.936038 31877 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:54.937119 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:54.942054 31725 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
W20260709 12:16:54.943069 31470 leader_election.cc:343] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.944922 31467 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:54.927184 31577 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:54.946516 31877 raft_consensus.cc:2804] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.946779 31691 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:54.947093 31877 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.947616 31877 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.950445 31400 catalog_manager.cc:5697] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.957000 31884 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:54.958556 31893 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.958983 31893 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.959328 31893 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.962726 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:54.963604 31577 raft_consensus.cc:2393] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:54.964109 31503 raft_consensus.cc:2393] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:54.967027 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:54.967662 31652 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.967593 31893 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:54.971225 31893 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:54.975122 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:54.975637 31577 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.977782 31807 consensus_queue.cc:1048] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:54.981397 31652 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:54.982872 31884 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:54.982968 31470 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:54.985016 31884 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.983697 31577 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:54.987725 31884 raft_consensus.cc:697] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:54.995865 31772 tablet_bootstrap.cc:654] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.997014 31691 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:55.002137 31906 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.002290 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:55.002539 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:54.995412 31884 consensus_queue.cc:237] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.003816 31893 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:55.004922 31690 leader_election.cc:343] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.010103 31806 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.008633 31877 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.023648 31727 raft_consensus.cc:1275] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:55.024310 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:55.025159 31652 raft_consensus.cc:2393] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:55.026196 31877 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.026327 31545 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:55.027665 31813 raft_consensus.cc:2749] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.027993 31776 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.028530 31776 ts_tablet_manager.cc:1403] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.153s	user 0.030s	sys 0.055s
I20260709 12:16:55.028671 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:55.029390 31727 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.030807 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:55.026072 31906 consensus_queue.cc:1048] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:55.032892 31470 leader_election.cc:343] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.036381 31776 raft_consensus.cc:359] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.037102 31776 raft_consensus.cc:385] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.037413 31776 raft_consensus.cc:740] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.020370 31893 raft_consensus.cc:697] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:55.039491 31776 consensus_queue.cc:260] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.020713 31806 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.044880 31727 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:55.045629 31776 ts_tablet_manager.cc:1434] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:16:55.046067 31467 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.046641 31776 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.047326 31877 raft_consensus.cc:2804] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.048043 31877 raft_consensus.cc:697] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:55.048880 31877 consensus_queue.cc:237] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.043642 31893 consensus_queue.cc:237] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.056138 31897 consensus_queue.cc:1048] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.053267 31776 tablet_bootstrap.cc:654] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.058993 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.065099 31806 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:55.074962 31771 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:55.075609 31771 ts_tablet_manager.cc:1403] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.165s	user 0.013s	sys 0.009s
I20260709 12:16:55.076730 31772 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:55.077237 31772 ts_tablet_manager.cc:1403] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.157s	user 0.019s	sys 0.000s
I20260709 12:16:55.077570 31774 tablet_bootstrap.cc:654] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.082372 31772 raft_consensus.cc:359] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.083210 31772 raft_consensus.cc:385] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.083564 31772 raft_consensus.cc:740] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.084218 31772 consensus_queue.cc:260] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.088217 31772 ts_tablet_manager.cc:1434] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:16:55.092604 31771 raft_consensus.cc:359] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.093400 31771 raft_consensus.cc:385] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.093708 31771 raft_consensus.cc:740] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.097970 31651 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.097996 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.093420 31398 catalog_manager.cc:5697] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.112342 31771 consensus_queue.cc:260] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.116259 31932 consensus_queue.cc:1048] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.117950 31771 ts_tablet_manager.cc:1434] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.003s
I20260709 12:16:55.118996 31771 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:55.119129 31616 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.129123 31772 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:55.136924 31771 tablet_bootstrap.cc:654] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.139127 31776 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.139633 31776 ts_tablet_manager.cc:1403] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.093s	user 0.012s	sys 0.009s
I20260709 12:16:55.147960 31772 tablet_bootstrap.cc:654] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.158051 31776 raft_consensus.cc:359] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.158849 31776 raft_consensus.cc:385] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.159153 31776 raft_consensus.cc:740] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.161609 31776 consensus_queue.cc:260] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.162813 31907 raft_consensus.cc:493] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.163373 31907 raft_consensus.cc:515] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.165019 31873 raft_consensus.cc:2749] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.165665 31907 leader_election.cc:290] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.166281 31774 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:55.166764 31774 ts_tablet_manager.cc:1403] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.240s	user 0.005s	sys 0.014s
I20260709 12:16:55.167384 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:55.168776 31542 leader_election.cc:343] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.156970 31400 catalog_manager.cc:5697] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.170828 31400 catalog_manager.cc:5697] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:55.172515 31774 raft_consensus.cc:359] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.173060 31774 raft_consensus.cc:385] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.173285 31774 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.175360 31884 consensus_queue.cc:1048] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.178606 31776 ts_tablet_manager.cc:1434] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:16:55.179987 31776 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.177847 31774 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.184252 31772 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:55.186378 31772 ts_tablet_manager.cc:1403] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.004s
I20260709 12:16:55.187701 31774 ts_tablet_manager.cc:1434] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:16:55.189311 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:55.190798 31545 leader_election.cc:343] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.191298 31545 leader_election.cc:304] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.192514 31906 raft_consensus.cc:2749] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.188762 31774 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:55.198084 31772 raft_consensus.cc:359] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.199124 31772 raft_consensus.cc:385] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.199487 31772 raft_consensus.cc:740] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.201686 31776 tablet_bootstrap.cc:654] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.203294 31401 catalog_manager.cc:5697] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:55.223153 31772 consensus_queue.cc:260] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.225792 31772 ts_tablet_manager.cc:1434] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:16:55.228015 31772 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:55.231510 31771 tablet_bootstrap.cc:492] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:55.232103 31771 ts_tablet_manager.cc:1403] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.113s	user 0.015s	sys 0.026s
I20260709 12:16:55.234961 31771 raft_consensus.cc:359] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.235683 31771 raft_consensus.cc:385] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.236061 31772 tablet_bootstrap.cc:654] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.235986 31771 raft_consensus.cc:740] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.237006 31771 consensus_queue.cc:260] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.239220 31774 tablet_bootstrap.cc:654] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.239711 31771 ts_tablet_manager.cc:1434] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:55.241005 31771 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:55.252820 31772 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:55.253382 31772 ts_tablet_manager.cc:1403] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.006s
I20260709 12:16:55.256358 31771 tablet_bootstrap.cc:654] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.259101 31906 raft_consensus.cc:493] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.259591 31906 raft_consensus.cc:515] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.259855 31772 raft_consensus.cc:359] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.260555 31772 raft_consensus.cc:385] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.259883 31776 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.260846 31772 raft_consensus.cc:740] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.261252 31776 ts_tablet_manager.cc:1403] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.082s	user 0.010s	sys 0.019s
I20260709 12:16:55.261781 31906 leader_election.cc:290] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.267431 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.268298 31501 raft_consensus.cc:2468] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:55.270401 31542 leader_election.cc:304] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:55.271250 31906 raft_consensus.cc:2804] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.271610 31906 raft_consensus.cc:493] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.272039 31906 raft_consensus.cc:3060] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.281628 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.282968 31873 raft_consensus.cc:493] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:55.283377 31545 leader_election.cc:343] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.283653 31873 raft_consensus.cc:515] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.287678 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.288750 31577 raft_consensus.cc:2393] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:55.290627 31806 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.291290 31806 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.305229 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.295984 31873 leader_election.cc:290] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.307092 31727 raft_consensus.cc:2393] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:55.305904 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6c957e6781843b9b69e2d98053daa74" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.303164 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.300074 31772 consensus_queue.cc:260] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.301625 31806 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:55.310920 31772 ts_tablet_manager.cc:1434] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 12:16:55.310662 31776 raft_consensus.cc:359] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.314899 31776 raft_consensus.cc:385] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.315351 31776 raft_consensus.cc:740] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.316818 31776 consensus_queue.cc:260] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.300421 31907 consensus_queue.cc:1048] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:16:55.320919 31776 ts_tablet_manager.cc:1434] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.059s	user 0.002s	sys 0.007s
I20260709 12:16:55.326308 31776 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.321794 31771 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:55.314081 31616 leader_election.cc:304] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.330870 31616 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.332311 31774 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:55.332652 31873 raft_consensus.cc:3060] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.332815 31774 ts_tablet_manager.cc:1403] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.144s	user 0.013s	sys 0.003s
I20260709 12:16:55.335101 31776 tablet_bootstrap.cc:654] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.336273 31808 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.337059 31774 raft_consensus.cc:359] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.337914 31774 raft_consensus.cc:385] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.338209 31774 raft_consensus.cc:740] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.339285 31774 consensus_queue.cc:260] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.312376 31772 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:55.341815 31774 ts_tablet_manager.cc:1434] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:55.342875 31774 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:55.316964 31906 raft_consensus.cc:515] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.347054 31906 leader_election.cc:290] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.348179 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:55.348851 31502 raft_consensus.cc:3060] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.349704 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:55.350667 31774 tablet_bootstrap.cc:654] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:55.351706 31545 leader_election.cc:343] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.356184 31808 raft_consensus.cc:2749] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.357216 31502 raft_consensus.cc:2468] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:55.358575 31873 raft_consensus.cc:2749] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.358780 31542 leader_election.cc:304] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.359615 31771 ts_tablet_manager.cc:1403] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.006s
I20260709 12:16:55.359901 31776 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.360014 31906 raft_consensus.cc:2804] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.360574 31776 ts_tablet_manager.cc:1403] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.004s
I20260709 12:16:55.360801 31906 raft_consensus.cc:697] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:55.361712 31906 consensus_queue.cc:237] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.362046 31774 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:55.362823 31774 ts_tablet_manager.cc:1403] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:16:55.363672 31776 raft_consensus.cc:359] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.364400 31776 raft_consensus.cc:385] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.364743 31776 raft_consensus.cc:740] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.365557 31774 raft_consensus.cc:359] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.366289 31774 raft_consensus.cc:385] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.366587 31774 raft_consensus.cc:740] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.367542 31774 consensus_queue.cc:260] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.365973 31776 consensus_queue.cc:260] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.370254 31776 ts_tablet_manager.cc:1434] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:55.375171 31776 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.362407 31771 raft_consensus.cc:359] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.375638 31774 ts_tablet_manager.cc:1434] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:55.375909 31771 raft_consensus.cc:385] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.376288 31771 raft_consensus.cc:740] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.376751 31774 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:55.383348 31774 tablet_bootstrap.cc:654] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.383746 31907 raft_consensus.cc:493] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.384271 31907 raft_consensus.cc:515] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.386543 31907 leader_election.cc:290] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:55.388355 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.388878 31776 tablet_bootstrap.cc:654] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:55.389895 31542 leader_election.cc:343] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.390820 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.391497 31502 raft_consensus.cc:2468] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:55.392729 31542 leader_election.cc:304] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.399838 31907 raft_consensus.cc:2804] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.404711 31907 raft_consensus.cc:493] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.405105 31907 raft_consensus.cc:3060] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.407755 31893 consensus_queue.cc:1048] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.408833 31776 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.409595 31776 ts_tablet_manager.cc:1403] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.001s
I20260709 12:16:55.413481 31776 raft_consensus.cc:359] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.415278 31893 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.418094 31893 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.418824 31776 raft_consensus.cc:385] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.419356 31776 raft_consensus.cc:740] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.420702 31893 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.428186 31776 consensus_queue.cc:260] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.431603 31776 ts_tablet_manager.cc:1434] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:16:55.421756 31772 tablet_bootstrap.cc:654] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.422694 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.433364 31776 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.413269 31907 raft_consensus.cc:515] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.423931 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.437808 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:55.438603 31932 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:55.438751 31691 leader_election.cc:343] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.439134 31932 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
W20260709 12:16:55.439996 31542 leader_election.cc:343] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.440083 31897 consensus_queue.cc:1048] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.439604 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:55.441205 31502 raft_consensus.cc:3060] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:55.442728 31693 leader_election.cc:343] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.443221 31693 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:55.443274 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.444895 31897 raft_consensus.cc:2749] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:55.444937 31690 leader_election.cc:343] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.445694 31932 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:55.445993 31776 tablet_bootstrap.cc:654] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.448087 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:55.450551 31691 leader_election.cc:343] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.450982 31691 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:55.451815 31932 raft_consensus.cc:2749] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.454088 31907 leader_election.cc:290] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:55.459877 31813 consensus_queue.cc:1048] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.460604 31501 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.463724 31771 consensus_queue.cc:260] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.465293 31776 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.465967 31776 ts_tablet_manager.cc:1403] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.000s
I20260709 12:16:55.469210 31771 ts_tablet_manager.cc:1434] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.000s
I20260709 12:16:55.470211 31771 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:55.469662 31776 raft_consensus.cc:359] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.474125 31776 raft_consensus.cc:385] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.474416 31776 raft_consensus.cc:740] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.475188 31776 consensus_queue.cc:260] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.481178 31774 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:55.481743 31774 ts_tablet_manager.cc:1403] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.105s	user 0.039s	sys 0.046s
I20260709 12:16:55.483049 31776 ts_tablet_manager.cc:1434] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:55.484012 31776 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.484822 31884 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.484897 31774 raft_consensus.cc:359] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.485700 31774 raft_consensus.cc:385] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.485862 31771 tablet_bootstrap.cc:654] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.485419 31884 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.492520 31502 raft_consensus.cc:2468] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:55.493731 31884 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:55.461848 31400 catalog_manager.cc:5697] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.494427 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.486020 31774 raft_consensus.cc:740] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.496814 31542 leader_election.cc:304] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.497048 31774 consensus_queue.cc:260] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.498232 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.488040 31932 consensus_queue.cc:1048] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.498868 31578 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:55.499866 31774 ts_tablet_manager.cc:1434] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:16:55.500237 31468 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.500942 31774 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:55.501549 31884 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:55.502668 31884 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.503089 31884 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.504459 31906 raft_consensus.cc:2804] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.505594 31906 raft_consensus.cc:697] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:55.489380 31776 tablet_bootstrap.cc:654] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.506794 31906 consensus_queue.cc:237] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.511737 31774 tablet_bootstrap.cc:654] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.517688 31877 consensus_queue.cc:1048] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.517845 31884 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.522917 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:55.524911 31502 raft_consensus.cc:1240] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Rejecting Update request from peer bf0f149240cc4f27b97810c90eba7ff9 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
W20260709 12:16:55.530149 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:55.531071 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:55.536532 31897 consensus_queue.cc:1059] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.537401 31897 raft_consensus.cc:3055] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.537757 31897 raft_consensus.cc:740] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:55.537933 31877 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.538511 31877 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.538581 31897 consensus_queue.cc:260] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.539978 31897 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.540827 31877 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.550804 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.552116 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.552776 31725 raft_consensus.cc:2468] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:55.533659 31884 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:55.547500 31577 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.555619 31467 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
W20260709 12:16:55.561576 31470 leader_election.cc:343] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.561805 31772 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:55.562326 31772 ts_tablet_manager.cc:1403] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.250s	user 0.012s	sys 0.001s
I20260709 12:16:55.564661 31884 consensus_queue.cc:1048] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:55.574038 31776 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.574548 31776 ts_tablet_manager.cc:1403] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.091s	user 0.012s	sys 0.003s
I20260709 12:16:55.564095 31877 raft_consensus.cc:2804] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.575946 31772 raft_consensus.cc:359] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.577358 31772 raft_consensus.cc:385] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.577716 31772 raft_consensus.cc:740] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.578197 31772 consensus_queue.cc:260] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.581615 31772 ts_tablet_manager.cc:1434] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.019s	user 0.001s	sys 0.005s
I20260709 12:16:55.576604 31877 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.586516 31877 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.591034 31807 consensus_queue.cc:1048] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.580376 31832 raft_consensus.cc:493] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.592219 31832 raft_consensus.cc:515] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.594367 31832 leader_election.cc:290] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.596143 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:55.597589 31468 leader_election.cc:343] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.599833 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.601332 31807 consensus_queue.cc:1048] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:55.601819 31470 leader_election.cc:343] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.602263 31470 leader_election.cc:304] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:55.603009 31771 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:55.603888 31771 ts_tablet_manager.cc:1403] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.134s	user 0.010s	sys 0.013s
I20260709 12:16:55.606779 31776 raft_consensus.cc:359] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.607581 31776 raft_consensus.cc:385] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.607846 31776 raft_consensus.cc:740] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.608026 31807 raft_consensus.cc:2749] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.608495 31776 consensus_queue.cc:260] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.610683 31776 ts_tablet_manager.cc:1434] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.001s
I20260709 12:16:55.611609 31776 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.611676 31577 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:16:55.613081 31468 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.616354 31774 tablet_bootstrap.cc:492] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:55.616847 31774 ts_tablet_manager.cc:1403] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.116s	user 0.016s	sys 0.010s
I20260709 12:16:55.623546 31772 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:55.625188 31771 raft_consensus.cc:359] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.640836 31771 raft_consensus.cc:385] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.641284 31771 raft_consensus.cc:740] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.621120 31884 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.653111 31771 consensus_queue.cc:260] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.654397 31884 raft_consensus.cc:697] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:55.666689 31884 consensus_queue.cc:237] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.621846 31877 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.671538 31877 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.672405 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:16:55.675525 31470 leader_election.cc:343] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.693914 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:55.694917 31725 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.688702 31774 raft_consensus.cc:359] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.721978 31774 raft_consensus.cc:385] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.722359 31776 tablet_bootstrap.cc:654] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.722424 31774 raft_consensus.cc:740] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.723526 31774 consensus_queue.cc:260] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.726342 31772 tablet_bootstrap.cc:654] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.727558 31774 ts_tablet_manager.cc:1434] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.110s	user 0.007s	sys 0.002s
W20260709 12:16:55.730962 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 47851.3us
I20260709 12:16:55.732587 31806 raft_consensus.cc:493] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.738590 31771 ts_tablet_manager.cc:1434] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.134s	user 0.009s	sys 0.000s
I20260709 12:16:55.739749 31771 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:55.733174 31806 raft_consensus.cc:515] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.775449 31978 raft_consensus.cc:493] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.774125 31808 raft_consensus.cc:493] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.775996 31978 raft_consensus.cc:515] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:55.783421 31808 raft_consensus.cc:515] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.791041 31776 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:55.791617 31776 ts_tablet_manager.cc:1403] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.180s	user 0.006s	sys 0.011s
I20260709 12:16:55.791467 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.792163 31577 raft_consensus.cc:2468] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:55.792260 31808 leader_election.cc:290] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.794055 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.794468 31776 raft_consensus.cc:359] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.794868 31503 raft_consensus.cc:2393] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:55.795728 31617 leader_election.cc:304] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:55.796327 31776 raft_consensus.cc:385] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.796638 31776 raft_consensus.cc:740] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.797402 31776 consensus_queue.cc:260] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.799587 31776 ts_tablet_manager.cc:1434] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:16:55.800539 31776 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:55.803586 31806 leader_election.cc:290] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:55.803855 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5f0372a71a4abe9b70c3772934163c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.805281 31873 raft_consensus.cc:2804] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.805749 31873 raft_consensus.cc:493] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.805920 31617 leader_election.cc:304] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:55.806123 31873 raft_consensus.cc:3060] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.807945 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.808779 31502 raft_consensus.cc:2393] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:55.811980 31725 raft_consensus.cc:2468] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:55.813717 31771 tablet_bootstrap.cc:654] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.813915 31873 raft_consensus.cc:515] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.813125 31400 catalog_manager.cc:5697] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:55.817991 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:55.817764 31402 catalog_manager.cc:5697] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.818594 31577 raft_consensus.cc:3060] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.820848 31774 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:55.822177 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.823136 31727 raft_consensus.cc:2468] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:55.825388 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c241197f69848f4b2142121633ec9ef" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:55.826021 31726 raft_consensus.cc:3060] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.829461 31806 raft_consensus.cc:3060] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.823428 31873 leader_election.cc:290] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.834254 31726 raft_consensus.cc:2468] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:55.837446 31616 leader_election.cc:304] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:55.838362 31980 raft_consensus.cc:2804] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.838868 31980 raft_consensus.cc:697] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:55.839710 31980 consensus_queue.cc:237] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:55.844408 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59dc1ab82c334d8f8954116a598b7c8b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:55.845760 31884 raft_consensus.cc:493] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.835269 31577 raft_consensus.cc:2468] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:55.846181 31691 leader_election.cc:304] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:55.854933 31467 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.856176 31807 raft_consensus.cc:2804] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:55.856665 31807 raft_consensus.cc:697] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:55.857558 31807 consensus_queue.cc:237] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.848120 31978 leader_election.cc:290] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:55.860672 31772 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:55.861208 31772 ts_tablet_manager.cc:1403] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.238s	user 0.012s	sys 0.003s
I20260709 12:16:55.860736 31932 raft_consensus.cc:3060] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.869603 31906 consensus_queue.cc:1048] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.874087 31932 raft_consensus.cc:2749] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.848758 31884 raft_consensus.cc:515] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.878185 31771 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:55.883795 31771 ts_tablet_manager.cc:1403] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.144s	user 0.003s	sys 0.013s
I20260709 12:16:55.889249 31884 leader_election.cc:290] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.889451 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.890635 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:55.891040 31470 leader_election.cc:343] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.890872 31397 catalog_manager.cc:5697] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:55.892536 31467 leader_election.cc:343] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.893056 31776 tablet_bootstrap.cc:654] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.893208 31467 leader_election.cc:304] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:55.894110 31772 raft_consensus.cc:359] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.894794 31772 raft_consensus.cc:385] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.895062 31772 raft_consensus.cc:740] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.895718 31772 consensus_queue.cc:260] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.900775 31771 raft_consensus.cc:359] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.904035 31906 raft_consensus.cc:493] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.906605 31906 raft_consensus.cc:515] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.907821 31774 tablet_bootstrap.cc:654] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.908754 31906 leader_election.cc:290] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.909324 31771 raft_consensus.cc:385] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:55.909670 31771 raft_consensus.cc:740] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:55.910287 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.910480 31771 consensus_queue.cc:260] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.904453 31813 consensus_queue.cc:1048] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.905344 31884 raft_consensus.cc:2749] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.906009 31807 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.915915 31807 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:16:55.916647 31545 leader_election.cc:343] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.916950 31772 ts_tablet_manager.cc:1434] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 12:16:55.918184 31771 ts_tablet_manager.cc:1434] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:16:55.918989 31772 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:55.918989 31771 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:55.904503 31806 raft_consensus.cc:2749] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:55.935879 31772 tablet_bootstrap.cc:654] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.941490 31906 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.942250 31906 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.946802 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:55.948743 31545 leader_election.cc:343] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.952047 31906 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.953646 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.954375 31503 raft_consensus.cc:2468] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:55.955432 31846 consensus_queue.cc:1048] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.958853 31542 leader_election.cc:304] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.960839 31846 raft_consensus.cc:2804] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.961304 31846 raft_consensus.cc:493] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.962023 31846 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.967365 31807 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:55.968463 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:55.968477 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:55.972015 31470 leader_election.cc:343] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.972054 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:55.973251 31503 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:55.974269 31807 raft_consensus.cc:493] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:55.974792 31807 raft_consensus.cc:515] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:55.975907 31846 raft_consensus.cc:515] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:55.977994 31807 leader_election.cc:290] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:55.977932 31542 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.983592 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:55.969380 31578 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:55.985502 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:55.985872 31813 raft_consensus.cc:2804] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:55.986156 31503 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.987001 31468 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:55.988339 31813 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:55.989876 31813 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:55.992134 31771 tablet_bootstrap.cc:654] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:55.989041 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:55.995839 31467 leader_election.cc:343] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:55.995911 31807 raft_consensus.cc:2804] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:16:56.004542 31470 leader_election.cc:343] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.005297 31397 catalog_manager.cc:5697] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.009755 31774 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.010262 31774 ts_tablet_manager.cc:1403] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.190s	user 0.020s	sys 0.001s
I20260709 12:16:56.012943 31774 raft_consensus.cc:359] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.013643 31774 raft_consensus.cc:385] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.013907 31774 raft_consensus.cc:740] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.014580 31774 consensus_queue.cc:260] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.007581 31470 leader_election.cc:304] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:56.018201 31877 raft_consensus.cc:2749] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.003494 31807 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.020280 31807 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.026901 31774 ts_tablet_manager.cc:1434] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:56.029222 31503 raft_consensus.cc:2468] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:56.030857 31776 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.034687 31776 ts_tablet_manager.cc:1403] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.234s	user 0.015s	sys 0.008s
I20260709 12:16:56.034988 31774 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.039266 31776 raft_consensus.cc:359] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.040285 31776 raft_consensus.cc:385] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.034391 31807 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.040712 31776 raft_consensus.cc:740] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.031683 31772 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.042052 31772 ts_tablet_manager.cc:1403] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.123s	user 0.012s	sys 0.001s
I20260709 12:16:56.043690 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:56.046211 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.046141 31772 raft_consensus.cc:359] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.046911 31772 raft_consensus.cc:385] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.047187 31772 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.047916 31772 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.048875 31776 consensus_queue.cc:260] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.051530 31776 ts_tablet_manager.cc:1434] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:16:56.052481 31776 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:16:56.053409 31545 leader_election.cc:343] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.054595 31846 leader_election.cc:290] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.057252 31807 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.057060 31542 leader_election.cc:304] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:56.064953 31906 raft_consensus.cc:2804] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.065595 31906 raft_consensus.cc:697] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:56.066654 31906 consensus_queue.cc:237] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.072655 31813 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.076565 31774 tablet_bootstrap.cc:654] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.079345 31813 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.076732 31907 consensus_queue.cc:1048] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.076565 31776 tablet_bootstrap.cc:654] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.077353 31772 ts_tablet_manager.cc:1434] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 12:16:56.081758 31578 raft_consensus.cc:2393] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:56.082865 31772 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.089097 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.090044 31502 raft_consensus.cc:2393] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:56.090334 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.091403 31772 tablet_bootstrap.cc:654] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:56.092254 31545 leader_election.cc:343] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.093266 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:16:56.095238 31470 leader_election.cc:343] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.096400 31468 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:56.097828 31877 raft_consensus.cc:2749] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.100229 31542 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:56.101161 31813 raft_consensus.cc:2749] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.116855 31906 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.117388 31906 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.118613 31776 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.119647 31776 ts_tablet_manager.cc:1403] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.010s
I20260709 12:16:56.122290 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.126363 31774 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.126752 31771 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.127234 31774 ts_tablet_manager.cc:1403] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.099s	user 0.008s	sys 0.017s
I20260709 12:16:56.127148 31776 raft_consensus.cc:359] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.127234 31771 ts_tablet_manager.cc:1403] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.208s	user 0.027s	sys 0.030s
I20260709 12:16:56.127956 31776 raft_consensus.cc:385] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.128288 31776 raft_consensus.cc:740] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.129016 31776 consensus_queue.cc:260] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.130514 31774 raft_consensus.cc:359] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.130805 31771 raft_consensus.cc:359] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.131302 31774 raft_consensus.cc:385] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.131556 31776 ts_tablet_manager.cc:1434] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:56.132001 31774 raft_consensus.cc:740] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
W20260709 12:16:56.132650 31545 leader_election.cc:343] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.132807 31776 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:56.132903 31774 consensus_queue.cc:260] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.133623 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.134303 31725 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:56.135617 31774 ts_tablet_manager.cc:1434] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:56.137465 31906 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.131531 31771 raft_consensus.cc:385] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.177945 31774 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.181582 31771 raft_consensus.cc:740] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.190912 31771 consensus_queue.cc:260] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.193418 31771 ts_tablet_manager.cc:1434] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 12:16:56.131425 31401 catalog_manager.cc:5697] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.185849 31772 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.194705 31772 ts_tablet_manager.cc:1403] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.112s	user 0.014s	sys 0.011s
I20260709 12:16:56.195089 31771 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.197508 31772 raft_consensus.cc:359] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.201987 31771 tablet_bootstrap.cc:654] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.203207 31807 raft_consensus.cc:493] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.203791 31807 raft_consensus.cc:515] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.135864 31542 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
W20260709 12:16:56.206117 31542 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 70361.6us
I20260709 12:16:56.206977 31807 leader_election.cc:290] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.198181 31772 raft_consensus.cc:385] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.217053 31772 raft_consensus.cc:740] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.217906 31772 consensus_queue.cc:260] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.218490 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.220057 31772 ts_tablet_manager.cc:1434] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 12:16:56.221045 31772 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.186297 31776 tablet_bootstrap.cc:654] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.207090 31813 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.221705 31813 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.222090 31813 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.187973 31774 tablet_bootstrap.cc:654] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.208069 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.228293 31776 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.228773 31776 ts_tablet_manager.cc:1403] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.096s	user 0.011s	sys 0.000s
W20260709 12:16:56.229236 31468 leader_election.cc:343] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.230105 31772 tablet_bootstrap.cc:654] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.231895 31776 raft_consensus.cc:359] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.232600 31776 raft_consensus.cc:385] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.232955 31776 raft_consensus.cc:740] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.233794 31776 consensus_queue.cc:260] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:16:56.236830 31467 leader_election.cc:343] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.237563 31776 ts_tablet_manager.cc:1434] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:16:56.238770 31776 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:56.240994 31772 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.241504 31772 ts_tablet_manager.cc:1403] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 12:16:56.244836 31776 tablet_bootstrap.cc:654] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.244565 31772 raft_consensus.cc:359] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.245255 31772 raft_consensus.cc:385] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.245585 31772 raft_consensus.cc:740] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.246403 31772 consensus_queue.cc:260] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.255139 31772 ts_tablet_manager.cc:1434] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:56.237329 31467 leader_election.cc:304] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:56.256448 31772 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.262017 31771 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.265089 31771 ts_tablet_manager.cc:1403] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.001s
I20260709 12:16:56.268466 31771 raft_consensus.cc:359] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.262713 31813 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.262544 31806 consensus_queue.cc:1048] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.279466 31813 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.279999 31771 raft_consensus.cc:385] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.280314 31771 raft_consensus.cc:740] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.280946 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:56.281032 31771 consensus_queue.cc:260] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.281664 31726 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.283303 31771 ts_tablet_manager.cc:1434] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:16:56.284294 31771 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.270258 31807 raft_consensus.cc:2749] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.271646 31772 tablet_bootstrap.cc:654] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.290144 31771 tablet_bootstrap.cc:654] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.272630 31776 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.276625 31774 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.288055 31832 consensus_queue.cc:1048] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.292054 31774 ts_tablet_manager.cc:1403] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.114s	user 0.010s	sys 0.007s
I20260709 12:16:56.292053 31776 ts_tablet_manager.cc:1403] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.053s	user 0.019s	sys 0.013s
I20260709 12:16:56.295598 31774 raft_consensus.cc:359] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.296275 31774 raft_consensus.cc:385] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.296541 31774 raft_consensus.cc:740] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.297206 31774 consensus_queue.cc:260] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.305191 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.305846 31726 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
W20260709 12:16:56.307134 31545 leader_election.cc:343] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.308429 31542 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:56.309656 31906 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.310170 31906 raft_consensus.cc:697] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:56.311060 31906 consensus_queue.cc:237] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.313864 31806 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.314353 31806 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.315016 31772 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.315541 31772 ts_tablet_manager.cc:1403] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.007s
I20260709 12:16:56.316335 31726 raft_consensus.cc:1275] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:16:56.316457 31806 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.316844 31726 pending_rounds.cc:85] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Aborting all ops after (but not including) 0
I20260709 12:16:56.317164 31726 pending_rounds.cc:107] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:16:56.317472 31726 raft_consensus.cc:2889] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:56.319033 31771 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.319563 31771 ts_tablet_manager.cc:1403] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.001s
I20260709 12:16:56.320633 31832 consensus_queue.cc:1048] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:56.322419 31771 raft_consensus.cc:359] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.323150 31771 raft_consensus.cc:385] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.323258 31978 raft_consensus.cc:493] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.323521 31771 raft_consensus.cc:740] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.323807 31978 raft_consensus.cc:515] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.325877 31978 leader_election.cc:290] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.309319 31776 raft_consensus.cc:359] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.326864 31776 raft_consensus.cc:385] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.326929 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.327224 31776 raft_consensus.cc:740] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.324368 31771 consensus_queue.cc:260] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.328413 31653 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.328532 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.329249 31502 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.330289 31693 leader_election.cc:304] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.330605 31774 ts_tablet_manager.cc:1434] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:16:56.331636 31774 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.331691 31897 raft_consensus.cc:2804] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.332525 31897 raft_consensus.cc:493] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.332966 31897 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.334110 31907 raft_consensus.cc:493] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.325856 31772 raft_consensus.cc:359] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.334673 31907 raft_consensus.cc:515] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.330652 31578 raft_consensus.cc:1275] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:56.334967 31772 raft_consensus.cc:385] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.336822 31907 leader_election.cc:290] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.336920 31772 raft_consensus.cc:740] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.337791 31772 consensus_queue.cc:260] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.339864 31772 ts_tablet_manager.cc:1434] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:16:56.339650 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.340626 31727 raft_consensus.cc:2468] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:56.340848 31772 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.342171 31542 leader_election.cc:304] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.342592 31807 consensus_queue.cc:1048] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.343034 31907 raft_consensus.cc:2804] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.343577 31907 raft_consensus.cc:493] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.343994 31907 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.346638 31806 raft_consensus.cc:493] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.348778 31772 tablet_bootstrap.cc:654] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.347309 31806 raft_consensus.cc:515] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.351781 31897 raft_consensus.cc:515] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.352262 31806 leader_election.cc:290] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.354166 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.347699 31776 consensus_queue.cc:260] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.357600 31776 ts_tablet_manager.cc:1434] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.001s
I20260709 12:16:56.359387 31776 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:56.362733 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.365659 31907 raft_consensus.cc:515] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.366238 31397 catalog_manager.cc:5697] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.367942 31907 leader_election.cc:290] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.363054 31774 tablet_bootstrap.cc:654] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.334360 31771 ts_tablet_manager.cc:1434] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 12:16:56.369364 31771 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.370272 31776 tablet_bootstrap.cc:654] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.380306 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:56.381531 31807 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.382211 31807 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.384871 31807 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.381210 31725 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.381191 31771 tablet_bootstrap.cc:654] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.398345 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.399307 31578 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:56.402163 31468 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:56.403659 31832 raft_consensus.cc:2804] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.404065 31832 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.404412 31832 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.406632 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.412948 31774 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.413147 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.413570 31774 ts_tablet_manager.cc:1403] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.000s
I20260709 12:16:56.413929 31653 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.415953 31725 raft_consensus.cc:2468] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:56.416435 31774 raft_consensus.cc:359] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.418907 31774 raft_consensus.cc:385] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.418972 31772 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.419344 31774 raft_consensus.cc:740] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.420513 31542 leader_election.cc:304] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.420459 31774 consensus_queue.cc:260] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.421875 31832 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.424086 31774 ts_tablet_manager.cc:1434] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:56.424295 31832 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.419587 31772 ts_tablet_manager.cc:1403] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.079s	user 0.022s	sys 0.015s
I20260709 12:16:56.427520 31774 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.427933 31771 tablet_bootstrap.cc:492] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.428519 31771 ts_tablet_manager.cc:1403] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.002s
I20260709 12:16:56.429304 31772 raft_consensus.cc:359] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.430032 31772 raft_consensus.cc:385] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.430297 31772 raft_consensus.cc:740] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.430956 31772 consensus_queue.cc:260] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.431183 31771 raft_consensus.cc:359] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.425101 31907 raft_consensus.cc:2804] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.432016 31907 raft_consensus.cc:697] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:56.433144 31776 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.432883 31907 consensus_queue.cc:237] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.433470 31772 ts_tablet_manager.cc:1434] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:56.433807 31776 ts_tablet_manager.cc:1403] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.041s
I20260709 12:16:56.434479 31772 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.437569 31771 raft_consensus.cc:385] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.438267 31771 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.440111 31774 tablet_bootstrap.cc:654] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.439785 31771 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.441061 31772 tablet_bootstrap.cc:654] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.444531 31771 ts_tablet_manager.cc:1434] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:16:56.448100 31771 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.450364 31897 leader_election.cc:290] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.453768 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:56.453631 31776 raft_consensus.cc:359] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.474596 31776 raft_consensus.cc:385] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.474931 31776 raft_consensus.cc:740] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.454155 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:16:56.457242 31467 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 48610.4us
I20260709 12:16:56.458971 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.476043 31502 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.476495 31503 raft_consensus.cc:2393] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:56.479274 31616 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:56.480085 31806 raft_consensus.cc:2749] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.481460 31806 consensus_queue.cc:1048] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.481802 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.483371 31907 consensus_queue.cc:1048] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.483886 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
W20260709 12:16:56.484570 31545 leader_election.cc:343] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.461544 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.466919 31771 tablet_bootstrap.cc:654] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.473183 31774 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.486321 31774 ts_tablet_manager.cc:1403] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.060s	user 0.006s	sys 0.011s
I20260709 12:16:56.484504 31726 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.475625 31776 consensus_queue.cc:260] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.489116 31774 raft_consensus.cc:359] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.489960 31774 raft_consensus.cc:385] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.490273 31774 raft_consensus.cc:740] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.491060 31774 consensus_queue.cc:260] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.492405 31776 ts_tablet_manager.cc:1434] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 12:16:56.474596 31577 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.494014 31776 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:56.483625 31652 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.499653 31502 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:56.484643 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.422524 31653 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:56.493239 31774 ts_tablet_manager.cc:1434] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.000s
I20260709 12:16:56.502528 31774 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.506268 31693 leader_election.cc:304] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.507262 31577 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:56.507997 31774 tablet_bootstrap.cc:654] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.508770 31468 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:56.510064 31884 raft_consensus.cc:2804] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.510728 31884 raft_consensus.cc:697] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
W20260709 12:16:56.510825 31545 leader_election.cc:343] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.511452 31616 leader_election.cc:304] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:56.475633 31397 catalog_manager.cc:5697] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.511649 31884 consensus_queue.cc:237] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.513567 31897 raft_consensus.cc:2804] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.514319 31897 raft_consensus.cc:697] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:56.514984 31806 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.517834 31726 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:56.468674 31772 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.520022 31772 ts_tablet_manager.cc:1403] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.086s	user 0.020s	sys 0.005s
I20260709 12:16:56.522457 31776 tablet_bootstrap.cc:654] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.522835 31806 raft_consensus.cc:2749] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.515165 31897 consensus_queue.cc:237] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.531829 31771 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.532371 31771 ts_tablet_manager.cc:1403] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.000s
I20260709 12:16:56.537770 31807 consensus_queue.cc:1048] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:56.539651 31807 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.540091 31807 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.540409 31772 raft_consensus.cc:359] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.541113 31772 raft_consensus.cc:385] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.541370 31772 raft_consensus.cc:740] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.542248 31771 raft_consensus.cc:359] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.542948 31771 raft_consensus.cc:385] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.543354 31771 raft_consensus.cc:740] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.543918 31807 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.544179 31771 consensus_queue.cc:260] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.537478 31813 consensus_queue.cc:1048] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:56.548663 31771 ts_tablet_manager.cc:1434] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:16:56.550590 31771 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.550645 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.558282 31771 tablet_bootstrap.cc:654] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.560511 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:56.561985 31470 leader_election.cc:343] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.542070 31772 consensus_queue.cc:260] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.570030 31577 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:56.570729 31772 ts_tablet_manager.cc:1434] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.000s
I20260709 12:16:56.573014 31772 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.578522 31774 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.579056 31774 ts_tablet_manager.cc:1403] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.077s	user 0.007s	sys 0.010s
I20260709 12:16:56.582291 31774 raft_consensus.cc:359] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.583089 31774 raft_consensus.cc:385] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.583405 31774 raft_consensus.cc:740] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.584470 31772 tablet_bootstrap.cc:654] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.584256 31774 consensus_queue.cc:260] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.586514 31774 ts_tablet_manager.cc:1434] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:56.587738 31774 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.590495 31771 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.590999 31771 ts_tablet_manager.cc:1403] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.015s
I20260709 12:16:56.593127 31772 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.593657 31772 ts_tablet_manager.cc:1403] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.005s
I20260709 12:16:56.593755 31776 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.594352 31776 ts_tablet_manager.cc:1403] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.002s
I20260709 12:16:56.594445 31774 tablet_bootstrap.cc:654] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.593861 31771 raft_consensus.cc:359] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.595885 31771 raft_consensus.cc:385] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.596138 31771 raft_consensus.cc:740] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.596361 31772 raft_consensus.cc:359] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.596696 31771 consensus_queue.cc:260] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.597100 31772 raft_consensus.cc:385] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.597565 31772 raft_consensus.cc:740] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.598523 31771 ts_tablet_manager.cc:1434] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:56.598421 31772 consensus_queue.cc:260] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.599426 31771 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.600632 31772 ts_tablet_manager.cc:1434] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:56.601979 31772 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:56.603957 31771 tablet_bootstrap.cc:654] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.607187 31774 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:56.607686 31774 ts_tablet_manager.cc:1403] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.011s
I20260709 12:16:56.604871 31776 raft_consensus.cc:359] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.608338 31776 raft_consensus.cc:385] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.572387 31468 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
W20260709 12:16:56.660604 31468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 88152.3us
I20260709 12:16:56.669653 31807 raft_consensus.cc:2804] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.670691 31807 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.671237 31807 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.608733 31776 raft_consensus.cc:740] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.673410 31806 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.674677 31806 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.607749 31772 tablet_bootstrap.cc:654] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.689497 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.690734 31813 raft_consensus.cc:493] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.702889 31503 raft_consensus.cc:2393] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:16:56.703244 31813 raft_consensus.cc:515] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.691596 31897 raft_consensus.cc:493] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.692600 31907 raft_consensus.cc:493] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.699728 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.700343 31932 raft_consensus.cc:493] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.694783 31806 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.724550 31577 raft_consensus.cc:2393] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:56.726153 31978 raft_consensus.cc:493] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.727161 31978 raft_consensus.cc:515] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.731292 31932 raft_consensus.cc:515] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.734139 31808 raft_consensus.cc:493] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.713783 31897 raft_consensus.cc:515] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.734838 31808 raft_consensus.cc:515] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.747750 31897 leader_election.cc:290] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.750490 31932 leader_election.cc:290] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.713716 31907 raft_consensus.cc:515] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.749939 31771 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:56.754583 31771 ts_tablet_manager.cc:1403] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.002s
I20260709 12:16:56.761698 31813 leader_election.cc:290] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.762786 31978 leader_election.cc:290] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.764514 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.765556 31501 raft_consensus.cc:2468] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.766048 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.767297 31690 leader_election.cc:304] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.768656 31776 consensus_queue.cc:260] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
W20260709 12:16:56.770112 31693 leader_election.cc:343] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.771921 31776 ts_tablet_manager.cc:1434] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.177s	user 0.007s	sys 0.000s
I20260709 12:16:56.756198 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.757134 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.758328 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.754442 31774 raft_consensus.cc:359] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.759510 31808 leader_election.cc:290] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.773504 31503 raft_consensus.cc:2468] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.773669 31502 raft_consensus.cc:2468] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.773661 31774 raft_consensus.cc:385] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.755700 31907 leader_election.cc:290] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.774760 31774 raft_consensus.cc:740] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.768998 31932 raft_consensus.cc:2804] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.775995 31932 raft_consensus.cc:493] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.776513 31932 raft_consensus.cc:3060] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.778169 31690 leader_election.cc:304] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.779881 31690 leader_election.cc:304] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.781967 31774 consensus_queue.cc:260] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
W20260709 12:16:56.784386 31693 leader_election.cc:343] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.785969 31897 raft_consensus.cc:2804] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.786440 31897 raft_consensus.cc:493] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.786855 31897 raft_consensus.cc:3060] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.788022 31978 raft_consensus.cc:2804] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.788662 31978 raft_consensus.cc:493] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.789549 31978 raft_consensus.cc:3060] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:56.798544 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 60733.2us
I20260709 12:16:56.818585 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.819397 31652 raft_consensus.cc:2468] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:56.821048 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.821947 31503 raft_consensus.cc:2468] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:56.829509 31771 raft_consensus.cc:359] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.830461 31771 raft_consensus.cc:385] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:56.830008 31978 raft_consensus.cc:515] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.832684 31978 leader_election.cc:290] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.830787 31771 raft_consensus.cc:740] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:56.834858 31771 consensus_queue.cc:260] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.836772 31616 leader_election.cc:304] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:56.837973 31776 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:16:56.846124 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:56.846390 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.846961 31545 leader_election.cc:304] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:56.847153 31577 raft_consensus.cc:2468] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:56.848031 31846 raft_consensus.cc:2804] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.848515 31846 raft_consensus.cc:493] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.848960 31846 raft_consensus.cc:3060] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.849807 31401 catalog_manager.cc:5697] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:56.858179 31897 raft_consensus.cc:515] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.860476 31897 leader_election.cc:290] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.861534 31980 raft_consensus.cc:2804] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:56.862092 31980 raft_consensus.cc:493] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:56.862485 31980 raft_consensus.cc:3060] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.862866 31776 tablet_bootstrap.cc:654] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:56.871652 31771 ts_tablet_manager.cc:1434] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.111s	user 0.000s	sys 0.008s
I20260709 12:16:56.876092 31846 raft_consensus.cc:515] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.877090 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:16:56.882412 31693 leader_election.cc:343] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.884244 31846 leader_election.cc:290] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.886466 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.887598 31772 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:56.887820 31846 raft_consensus.cc:493] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.888561 31897 raft_consensus.cc:493] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.888365 31846 raft_consensus.cc:515] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.889214 31897 raft_consensus.cc:515] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.892768 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:56.892890 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.893528 31577 raft_consensus.cc:3060] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.894439 31806 raft_consensus.cc:493] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:56.893323 31846 leader_election.cc:290] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.895102 31806 raft_consensus.cc:515] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
W20260709 12:16:56.898195 31693 leader_election.cc:343] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.899322 31806 leader_election.cc:290] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:56.901772 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:56.904919 31616 leader_election.cc:343] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.902733 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:56.888415 31772 ts_tablet_manager.cc:1403] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.287s	user 0.024s	sys 0.005s
I20260709 12:16:56.912999 31771 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:56.914875 31897 leader_election.cc:290] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.916566 31980 raft_consensus.cc:515] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.916921 32038 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:56.909809 31690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 130089us
I20260709 12:16:56.918226 31774 ts_tablet_manager.cc:1434] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.310s	user 0.013s	sys 0.026s
W20260709 12:16:56.921000 31545 leader_election.cc:343] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.921022 31653 raft_consensus.cc:2393] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:56.923391 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.923321 32038 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:56.924223 31501 raft_consensus.cc:3060] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.930204 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:56.931960 31693 leader_election.cc:343] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.933257 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.919495 31577 raft_consensus.cc:2468] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:56.933943 31502 raft_consensus.cc:3060] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.928802 32038 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.939263 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.914327 31932 raft_consensus.cc:515] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:56.941250 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:56.942148 31727 raft_consensus.cc:2393] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:56.942508 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "506b4da2b0be46d5bf3ff52167d4419b" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.942142 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:56.943738 31503 raft_consensus.cc:2468] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:56.944334 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:56.945497 31691 leader_election.cc:304] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:56.947731 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:56.950311 32041 raft_consensus.cc:2804] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:56.955775 32041 raft_consensus.cc:697] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:56.948647 31652 raft_consensus.cc:3060] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.960150 31771 tablet_bootstrap.cc:654] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:56.960485 31691 leader_election.cc:343] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.961023 31691 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:56.962277 31897 raft_consensus.cc:2749] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:56.960772 32041 consensus_queue.cc:237] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:56.962776 31980 leader_election.cc:290] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:56.963627 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.964325 31503 raft_consensus.cc:3060] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.968847 31774 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:56.967553 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.973726 31932 leader_election.cc:290] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:56.936499 31807 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:56.987360 31502 raft_consensus.cc:2468] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:56.975175 31501 raft_consensus.cc:2468] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
W20260709 12:16:56.992259 31690 leader_election.cc:343] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:56.993167 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc5b5e283ec34f03b184be6dfa4e046e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:56.993633 31776 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:56.994002 31501 raft_consensus.cc:3060] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:56.994292 31776 ts_tablet_manager.cc:1403] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.157s	user 0.011s	sys 0.006s
I20260709 12:16:57.000553 32034 consensus_queue.cc:1048] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:56.992158 31397 catalog_manager.cc:5697] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.002555 31653 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.007261 31807 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:56.993198 31690 leader_election.cc:304] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:57.008073 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:16:57.008562 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 274384us
W20260709 12:16:57.011802 31470 leader_election.cc:343] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.012472 31690 leader_election.cc:304] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:57.014899 31897 raft_consensus.cc:2749] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.015822 31576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.016251 32038 raft_consensus.cc:2804] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.016767 31576 raft_consensus.cc:2393] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:57.016825 32038 raft_consensus.cc:697] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:57.016656 31617 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:57.018085 32038 consensus_queue.cc:237] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.020722 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.000002 31776 raft_consensus.cc:359] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.034435 31776 raft_consensus.cc:385] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.035445 31776 raft_consensus.cc:740] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.037252 31776 consensus_queue.cc:260] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.035780 31806 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.050462 31776 ts_tablet_manager.cc:1434] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.055s	user 0.004s	sys 0.008s
I20260709 12:16:57.051113 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.052620 31772 raft_consensus.cc:359] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.054317 31772 raft_consensus.cc:385] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.054571 31772 raft_consensus.cc:740] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.057999 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:57.058673 31574 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.061919 32034 raft_consensus.cc:493] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.064872 31503 raft_consensus.cc:2468] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:57.077597 32034 raft_consensus.cc:515] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.080597 32034 leader_election.cc:290] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:57.081135 31616 leader_election.cc:304] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:16:57.082232 31652 raft_consensus.cc:2468] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:57.047734 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ad0edffb3874e40994f98fd888a00b3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:57.084282 31575 raft_consensus.cc:2393] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:57.058962 31501 raft_consensus.cc:2468] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:57.061527 31772 consensus_queue.cc:260] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.087572 31690 leader_election.cc:304] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.091207 32036 raft_consensus.cc:2804] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.091470 31873 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.091888 32036 raft_consensus.cc:697] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:57.092056 31873 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.094501 31932 raft_consensus.cc:2804] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:57.089735 31617 leader_election.cc:343] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.089877 31574 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:16:57.101534 31772 ts_tablet_manager.cc:1434] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.192s	user 0.007s	sys 0.001s
I20260709 12:16:57.092964 32036 consensus_queue.cc:237] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.099025 31932 raft_consensus.cc:697] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:57.109212 31776 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:57.110296 31771 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:57.110725 31771 ts_tablet_manager.cc:1403] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.199s	user 0.016s	sys 0.009s
I20260709 12:16:57.106154 31617 leader_election.cc:304] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.121667 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.118060 32037 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:57.127034 31542 leader_election.cc:304] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:57.128605 32041 consensus_queue.cc:1048] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:57.139690 31772 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
W20260709 12:16:57.151456 31467 leader_election.cc:343] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.143224 32032 raft_consensus.cc:2749] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.150252 31873 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
W20260709 12:16:57.164671 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 74989.8us
I20260709 12:16:57.164834 31980 raft_consensus.cc:2749] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.148542 31806 raft_consensus.cc:2749] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.174860 31771 raft_consensus.cc:359] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.175628 31771 raft_consensus.cc:385] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.175998 31771 raft_consensus.cc:740] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.176791 31771 consensus_queue.cc:260] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.146539 32038 consensus_queue.cc:1048] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260709 12:16:57.179790 31932 consensus_queue.cc:237] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.179699 31397 catalog_manager.cc:5697] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.184290 32032 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:57.185663 31542 leader_election.cc:304] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.187207 31774 tablet_bootstrap.cc:654] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.187834 31771 ts_tablet_manager.cc:1434] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
I20260709 12:16:57.189043 31772 tablet_bootstrap.cc:654] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.200583 31771 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:57.205492 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.206427 31727 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
W20260709 12:16:57.189368 31545 consensus_peers.cc:597] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.201759 31397 catalog_manager.cc:5697] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.210304 31846 raft_consensus.cc:2749] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.227874 31776 tablet_bootstrap.cc:654] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.233866 31771 tablet_bootstrap.cc:654] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.240283 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.262228 31616 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:57.263283 31873 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:57.263775 31873 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 843512b2aa644b55a21b7e702943e02f)
I20260709 12:16:57.264190 31873 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:57.240521 31542 leader_election.cc:343] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.280956 31772 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:57.281589 31772 ts_tablet_manager.cc:1403] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.142s	user 0.009s	sys 0.009s
I20260709 12:16:57.284689 31772 raft_consensus.cc:359] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.285353 31772 raft_consensus.cc:385] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.285646 31772 raft_consensus.cc:740] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.286322 31772 consensus_queue.cc:260] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.288008 31776 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:57.288493 31776 ts_tablet_manager.cc:1403] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.189s	user 0.012s	sys 0.023s
I20260709 12:16:57.277074 31542 leader_election.cc:304] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.292097 31776 raft_consensus.cc:359] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.291215 31772 ts_tablet_manager.cc:1434] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:57.292847 31776 raft_consensus.cc:385] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.293687 31776 raft_consensus.cc:740] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.293416 31813 raft_consensus.cc:2749] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.301105 31772 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:57.303829 31873 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.313345 31776 consensus_queue.cc:260] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
W20260709 12:16:57.307379 31542 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 66918.1us
I20260709 12:16:57.314514 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:57.316457 31651 raft_consensus.cc:1240] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Rejecting Update request from peer 843512b2aa644b55a21b7e702943e02f for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:57.318321 31468 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:57.320116 32033 consensus_queue.cc:1048] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:57.313481 31397 catalog_manager.cc:5697] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.321380 32034 raft_consensus.cc:2804] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.321915 32034 raft_consensus.cc:697] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:57.322844 32034 consensus_queue.cc:237] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.329996 31873 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:57.329419 31813 consensus_queue.cc:1048] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:57.332243 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.333281 32032 raft_consensus.cc:493] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.333784 32032 raft_consensus.cc:515] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.335994 32032 leader_election.cc:290] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:57.341305 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.342942 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:57.343631 31500 raft_consensus.cc:2468] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
W20260709 12:16:57.345331 31468 leader_election.cc:343] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.345836 31468 leader_election.cc:304] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.346858 32033 raft_consensus.cc:2749] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.349856 31542 leader_election.cc:304] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:57.351186 32032 raft_consensus.cc:2804] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:57.351548 32032 raft_consensus.cc:493] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:57.351962 32032 raft_consensus.cc:3060] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:57.353451 31542 leader_election.cc:343] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.358903 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:57.367471 31616 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.369141 31776 ts_tablet_manager.cc:1434] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.002s
I20260709 12:16:57.370455 31776 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:57.369022 31873 raft_consensus.cc:2749] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:57.374677 31397 catalog_manager.cc:5697] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:57.395620 31774 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:57.396219 31774 ts_tablet_manager.cc:1403] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.429s	user 0.015s	sys 0.005s
I20260709 12:16:57.372514 31846 consensus_queue.cc:1059] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:16:57.397670 31906 raft_consensus.cc:3055] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:57.398046 31906 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:57.398885 31906 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.400120 31906 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:57.403679 31772 tablet_bootstrap.cc:654] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.408308 31774 raft_consensus.cc:359] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.409086 31774 raft_consensus.cc:385] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.409507 31774 raft_consensus.cc:740] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.412475 31774 consensus_queue.cc:260] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.413863 31813 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.430166 31774 ts_tablet_manager.cc:1434] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.002s
I20260709 12:16:57.425307 31813 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.435031 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:57.435765 31771 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:57.436256 31771 ts_tablet_manager.cc:1403] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.236s	user 0.025s	sys 0.013s
W20260709 12:16:57.436954 31545 leader_election.cc:343] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.442736 31774 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:57.448397 31813 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:57.460937 32032 raft_consensus.cc:515] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.462198 31771 raft_consensus.cc:359] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.463079 31771 raft_consensus.cc:385] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.463289 31771 raft_consensus.cc:740] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.471480 31774 tablet_bootstrap.cc:654] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.471256 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:57.474972 31897 consensus_queue.cc:1048] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:57.481954 31542 leader_election.cc:343] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.485281 31542 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:57.487253 32037 raft_consensus.cc:2749] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:57.487366 32032 leader_election.cc:290] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:57.484302 31771 consensus_queue.cc:260] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.492038 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:57.499241 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:57.499927 31501 raft_consensus.cc:3060] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.502455 31776 tablet_bootstrap.cc:654] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.488531 32033 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.503593 32033 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.505846 32033 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:57.507601 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:57.509120 31470 leader_election.cc:343] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.509980 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:57.511479 31467 leader_election.cc:343] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.511909 31467 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.517581 32033 raft_consensus.cc:2749] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:57.523669 31542 leader_election.cc:343] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.517280 31501 raft_consensus.cc:2468] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:57.532215 31542 leader_election.cc:304] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:57.533042 31906 raft_consensus.cc:2804] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.533380 31774 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:57.535550 31774 ts_tablet_manager.cc:1403] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.016s
I20260709 12:16:57.533716 31906 raft_consensus.cc:697] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:57.541533 31771 ts_tablet_manager.cc:1434] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.105s	user 0.007s	sys 0.001s
I20260709 12:16:57.544886 31771 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:57.546921 31906 consensus_queue.cc:237] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.548997 31772 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:57.549530 31772 ts_tablet_manager.cc:1403] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.249s	user 0.019s	sys 0.016s
I20260709 12:16:57.552285 31776 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:57.552781 31776 ts_tablet_manager.cc:1403] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.183s	user 0.011s	sys 0.010s
I20260709 12:16:57.550472 31400 catalog_manager.cc:5697] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.558884 31776 raft_consensus.cc:359] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.559561 31776 raft_consensus.cc:385] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.559834 31776 raft_consensus.cc:740] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.560521 31776 consensus_queue.cc:260] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.562472 31776 ts_tablet_manager.cc:1434] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:16:57.563575 31776 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:57.564507 31772 raft_consensus.cc:359] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.565232 31772 raft_consensus.cc:385] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.565538 31772 raft_consensus.cc:740] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.568199 31774 raft_consensus.cc:359] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.568938 31774 raft_consensus.cc:385] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.569201 31774 raft_consensus.cc:740] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.566295 31772 consensus_queue.cc:260] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.578003 31776 tablet_bootstrap.cc:654] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.578495 31774 consensus_queue.cc:260] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.579500 31772 ts_tablet_manager.cc:1434] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 12:16:57.579596 32041 consensus_queue.cc:1048] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:57.580637 31772 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:57.577562 31771 tablet_bootstrap.cc:654] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.595077 31774 ts_tablet_manager.cc:1434] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.059s	user 0.003s	sys 0.004s
I20260709 12:16:57.598158 31774 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:57.604467 31774 tablet_bootstrap.cc:654] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.606442 31772 tablet_bootstrap.cc:654] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.617725 31776 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:57.618388 31776 ts_tablet_manager.cc:1403] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.005s
I20260709 12:16:57.618733 31774 tablet_bootstrap.cc:492] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:57.619185 31774 ts_tablet_manager.cc:1403] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.005s
I20260709 12:16:57.621802 31774 raft_consensus.cc:359] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.622483 31774 raft_consensus.cc:385] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.622741 31774 raft_consensus.cc:740] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.623579 31774 consensus_queue.cc:260] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.624854 31776 raft_consensus.cc:359] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.626343 31776 raft_consensus.cc:385] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.626693 31776 raft_consensus.cc:740] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.627476 31776 consensus_queue.cc:260] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.629673 31400 catalog_manager.cc:5697] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:57.633745 31774 ts_tablet_manager.cc:1434] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:16:57.634485 31806 raft_consensus.cc:493] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.634826 31774 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:57.635051 31806 raft_consensus.cc:515] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.637410 31806 leader_election.cc:290] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:57.639207 31980 consensus_queue.cc:1048] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:16:57.646955 31873 raft_consensus.cc:493] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.648716 31873 raft_consensus.cc:515] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.650906 31873 leader_election.cc:290] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:57.656183 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.656975 31727 raft_consensus.cc:2468] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:57.657275 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:57.658212 31723 raft_consensus.cc:2468] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:16:57.658418 31616 leader_election.cc:304] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.658545 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:57.660987 31873 raft_consensus.cc:2804] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:57.661439 31873 raft_consensus.cc:493] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:57.661870 31873 raft_consensus.cc:3060] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.667088 31771 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:57.667663 31771 ts_tablet_manager.cc:1403] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.123s	user 0.008s	sys 0.004s
I20260709 12:16:57.670694 31771 raft_consensus.cc:359] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.671423 31771 raft_consensus.cc:385] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.671708 31771 raft_consensus.cc:740] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.672443 31771 consensus_queue.cc:260] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.673709 31616 leader_election.cc:304] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.675628 31980 raft_consensus.cc:2804] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:57.675987 31980 raft_consensus.cc:493] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:57.676303 31980 raft_consensus.cc:3060] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.684451 31980 raft_consensus.cc:515] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
W20260709 12:16:57.685377 31616 leader_election.cc:343] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.686506 31980 leader_election.cc:290] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:57.687304 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:57.688009 31574 raft_consensus.cc:3060] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.688580 31772 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:57.689095 31772 ts_tablet_manager.cc:1403] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.109s	user 0.015s	sys 0.006s
I20260709 12:16:57.692127 31772 raft_consensus.cc:359] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.692821 31772 raft_consensus.cc:385] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.693100 31772 raft_consensus.cc:740] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.694145 31772 consensus_queue.cc:260] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.695771 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:57.696403 31723 raft_consensus.cc:3060] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.698839 31771 ts_tablet_manager.cc:1434] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 12:16:57.700065 31771 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:57.644336 32041 consensus_queue.cc:1048] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.057s
I20260709 12:16:57.645262 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a44cca173ddc49d99e53c40bccf8b5c9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.643672 31774 tablet_bootstrap.cc:654] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.704975 31723 raft_consensus.cc:2468] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:57.706394 31897 raft_consensus.cc:493] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.706900 31897 raft_consensus.cc:515] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.707324 32036 consensus_queue.cc:1048] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:57.708072 31693 consensus_peers.cc:597] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:57.710420 31616 leader_election.cc:304] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.711268 32077 raft_consensus.cc:2804] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.711766 32077 raft_consensus.cc:697] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:57.712579 32077 consensus_queue.cc:237] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.713909 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:57.701395 31772 ts_tablet_manager.cc:1434] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:16:57.714572 31501 raft_consensus.cc:2468] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:57.715570 31772 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:57.719033 31690 leader_election.cc:304] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.720021 32041 raft_consensus.cc:2804] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:57.720374 32041 raft_consensus.cc:493] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:57.720709 32041 raft_consensus.cc:3060] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.722985 31574 raft_consensus.cc:2468] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:57.728219 31897 leader_election.cc:290] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:57.728992 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.733551 32033 consensus_queue.cc:1048] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:57.734011 31691 leader_election.cc:343] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.728940 31873 raft_consensus.cc:515] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.737504 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:57.738216 31723 raft_consensus.cc:3060] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:57.742610 31897 consensus_queue.cc:1048] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:57.714596 31776 ts_tablet_manager.cc:1434] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.096s	user 0.004s	sys 0.001s
I20260709 12:16:57.751173 31723 raft_consensus.cc:2468] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:16:57.755242 31776 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:57.772810 31616 leader_election.cc:304] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:57.773659 31771 tablet_bootstrap.cc:654] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.773908 31873 leader_election.cc:290] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:57.774471 31873 raft_consensus.cc:2804] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:57.775056 31873 raft_consensus.cc:697] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:16:57.775880 31873 consensus_queue.cc:237] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.776376 31774 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:57.776881 31774 ts_tablet_manager.cc:1403] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.000s
I20260709 12:16:57.779048 32033 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.779583 32033 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:57.779733 31774 raft_consensus.cc:359] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.780450 31774 raft_consensus.cc:385] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.780849 31774 raft_consensus.cc:740] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.785596 31772 tablet_bootstrap.cc:654] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.784444 31774 consensus_queue.cc:260] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.767830 31932 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.763674 32041 raft_consensus.cc:515] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:57.797180 31776 tablet_bootstrap.cc:654] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.799428 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:57.801574 31401 catalog_manager.cc:5697] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:57.816144 31774 ts_tablet_manager.cc:1434] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:16:57.818470 31774 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:57.826964 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:16:57.836077 31616 leader_election.cc:343] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.847445 31774 tablet_bootstrap.cc:654] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:57.794173 31932 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:57.810995 31574 raft_consensus.cc:2468] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:57.846381 32033 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:57.883458 31771 tablet_bootstrap.cc:492] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:57.883997 31771 ts_tablet_manager.cc:1403] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.184s	user 0.018s	sys 0.038s
I20260709 12:16:57.895957 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:57.885105 32033 raft_consensus.cc:493] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.899991 31932 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:57.940726 31771 raft_consensus.cc:359] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:57.941298 31771 raft_consensus.cc:385] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:57.941598 31771 raft_consensus.cc:740] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:57.925508 32033 raft_consensus.cc:515] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:16:57.932058 31468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 67835.6us
I20260709 12:16:57.950416 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:57.938618 31980 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.969918 31771 consensus_queue.cc:260] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
W20260709 12:16:57.940451 31470 leader_election.cc:343] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:57.930720 32038 consensus_queue.cc:1048] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 12:16:57.946805 32041 leader_election.cc:290] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
W20260709 12:16:57.987295 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 47896.9us
I20260709 12:16:57.974486 31906 raft_consensus.cc:493] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.989638 31813 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:57.969202 31980 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.006945 31771 ts_tablet_manager.cc:1434] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.123s	user 0.007s	sys 0.002s
I20260709 12:16:58.014935 31771 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:58.015105 31980 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:58.017309 31776 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:58.017841 31776 ts_tablet_manager.cc:1403] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.263s	user 0.016s	sys 0.009s
I20260709 12:16:58.018136 31774 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:58.018621 31774 ts_tablet_manager.cc:1403] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.200s	user 0.016s	sys 0.016s
I20260709 12:16:58.020622 31776 raft_consensus.cc:359] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.002959 31906 raft_consensus.cc:515] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.023697 31906 leader_election.cc:290] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.023430 31813 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.025599 31813 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:58.027488 32033 leader_election.cc:290] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.027923 31771 tablet_bootstrap.cc:654] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:58.030256 31693 leader_election.cc:343] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.021078 31774 raft_consensus.cc:359] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.021381 31776 raft_consensus.cc:385] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.035892 31776 raft_consensus.cc:740] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.035784 31774 raft_consensus.cc:385] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.036463 31774 raft_consensus.cc:740] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.036450 31772 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:58.037897 31772 ts_tablet_manager.cc:1403] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.323s	user 0.017s	sys 0.010s
I20260709 12:16:58.039048 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:58.039963 31727 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:58.040056 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.041301 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.049142 31651 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.051999 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.052171 31774 consensus_queue.cc:260] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.054849 31774 ts_tablet_manager.cc:1434] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 12:16:58.055962 31774 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
W20260709 12:16:58.056928 31470 leader_election.cc:343] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.058076 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.058480 32037 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.058739 31501 raft_consensus.cc:2468] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:16:58.070016 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.068084 31776 consensus_queue.cc:260] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.070892 31501 raft_consensus.cc:2468] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
W20260709 12:16:58.072350 32084 log.cc:927] Time spent T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Append to log took a long time: real 0.195s	user 0.004s	sys 0.001s
I20260709 12:16:58.073051 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:58.073709 31501 raft_consensus.cc:3060] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.092615 31468 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:58.094399 31542 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:58.095569 32032 raft_consensus.cc:2804] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.095796 31616 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:58.096182 32032 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.096807 32032 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.056639 31401 catalog_manager.cc:5697] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.122486 31776 ts_tablet_manager.cc:1434] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.104s	user 0.012s	sys 0.000s
I20260709 12:16:58.126263 31832 raft_consensus.cc:2804] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.126686 31832 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.127033 31832 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.127434 31776 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:58.122145 32037 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.136808 32037 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.139333 32077 consensus_queue.cc:1048] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:58.141937 31772 raft_consensus.cc:359] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.142632 31772 raft_consensus.cc:385] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.142897 31772 raft_consensus.cc:740] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.143738 31772 consensus_queue.cc:260] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
W20260709 12:16:58.105903 31545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 95265.8us
I20260709 12:16:58.146584 31772 ts_tablet_manager.cc:1434] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.108s	user 0.003s	sys 0.006s
I20260709 12:16:58.147606 31772 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:58.148008 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.148712 31649 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:58.151611 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45a8924a0b9c4dbc9ace49508d7b9f7e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:58.152201 31579 raft_consensus.cc:3060] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.154239 31980 raft_consensus.cc:2749] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:58.155546 32041 raft_consensus.cc:493] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.156020 32041 raft_consensus.cc:515] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.158080 32041 leader_election.cc:290] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:58.158457 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.160939 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.161617 32095 consensus_queue.cc:1048] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:58.164172 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:58.164355 31579 raft_consensus.cc:2468] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:58.166419 31542 leader_election.cc:304] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:58.166607 32032 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.167904 31906 raft_consensus.cc:2804] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.168375 31906 raft_consensus.cc:493] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.168756 31906 raft_consensus.cc:3060] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.173481 32032 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
W20260709 12:16:58.160081 31468 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 66723.6us
I20260709 12:16:58.180794 31651 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.187322 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:58.189302 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:58.189919 31723 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.193921 31906 raft_consensus.cc:515] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.195967 31906 leader_election.cc:290] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.197006 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.196974 31690 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:58.200174 32090 raft_consensus.cc:2804] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.200562 32090 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.200883 32090 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.204630 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.205329 31504 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:58.206472 31545 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
W20260709 12:16:58.207485 31545 leader_election.cc:343] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:58.208926 31545 leader_election.cc:343] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:58.209964 31545 leader_election.cc:343] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.207964 31906 raft_consensus.cc:2804] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:58.210533 31906 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.210909 31906 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.212314 31980 consensus_queue.cc:1048] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:58.216410 32037 consensus_queue.cc:1048] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:58.218189 32090 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.219967 32077 consensus_queue.cc:1048] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:58.220511 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:58.221026 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:16:58.222339 31693 leader_election.cc:343] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.223242 31906 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:16:58.225208 31545 leader_election.cc:343] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.225692 32098 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.226199 32098 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.227070 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:58.227702 31649 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.228308 32098 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.229967 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:58.230569 31504 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.228339 31499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "384f88a7446842bfb4a6c71e921c9049" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.225232 31906 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.231554 31499 raft_consensus.cc:2393] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:58.232512 31776 tablet_bootstrap.cc:654] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:58.233533 31691 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 47203.2us
I20260709 12:16:58.234398 31691 leader_election.cc:304] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:58.241016 31772 tablet_bootstrap.cc:654] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.225986 32090 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:58.235020 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.252635 31932 raft_consensus.cc:2804] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:58.255952 31932 raft_consensus.cc:697] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:58.256716 31774 tablet_bootstrap.cc:654] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.256824 31932 consensus_queue.cc:237] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.253145 31649 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:16:58.254829 31771 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:58.259907 31504 raft_consensus.cc:2468] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:58.261284 31771 ts_tablet_manager.cc:1403] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.247s	user 0.021s	sys 0.003s
I20260709 12:16:58.261632 31499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:58.264168 31771 raft_consensus.cc:359] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.264834 31771 raft_consensus.cc:385] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.265080 31771 raft_consensus.cc:740] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.183135 31832 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.262256 31499 raft_consensus.cc:3060] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.267560 31832 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.267405 31771 consensus_queue.cc:260] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
W20260709 12:16:58.286216 31542 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 69884us
I20260709 12:16:58.289139 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.290748 31579 raft_consensus.cc:2393] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:58.297451 31576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.297796 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:58.298303 31576 raft_consensus.cc:2393] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 2: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:58.298461 31577 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.328779 31771 ts_tablet_manager.cc:1434] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:16:58.280728 31545 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:58.280977 31691 leader_election.cc:304] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:58.346537 31906 raft_consensus.cc:2804] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:58.347051 31906 raft_consensus.cc:697] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:58.352461 31771 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:58.355007 31501 raft_consensus.cc:2468] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:58.360131 31771 tablet_bootstrap.cc:654] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:58.363046 31691 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 88679.3us
I20260709 12:16:58.334035 31690 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:58.336562 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:58.372238 31906 consensus_queue.cc:237] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.369712 31468 leader_election.cc:304] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:58.374876 32085 raft_consensus.cc:2804] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:58.375419 32085 raft_consensus.cc:697] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:58.376215 32085 consensus_queue.cc:237] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.377138 31468 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:58.379978 32098 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.390522 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:58.369524 32092 raft_consensus.cc:3060] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:58.351822 31545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 76672.4us
I20260709 12:16:58.373380 32038 raft_consensus.cc:493] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.424341 31499 raft_consensus.cc:2468] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:58.427281 31577 raft_consensus.cc:2468] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
W20260709 12:16:58.429390 32104 log.cc:927] Time spent T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:16:58.431521 32091 raft_consensus.cc:493] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:58.406656 31690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 76884.1us
I20260709 12:16:58.432219 32038 raft_consensus.cc:515] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.445058 32091 raft_consensus.cc:515] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.435016 31832 raft_consensus.cc:2749] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:58.450836 32038 leader_election.cc:290] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.445058 31723 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:16:58.454131 31542 leader_election.cc:304] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
W20260709 12:16:58.465971 32107 log.cc:927] Time spent T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:16:58.467594 31504 raft_consensus.cc:3060] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.469792 31776 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:58.470350 31776 ts_tablet_manager.cc:1403] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.344s	user 0.016s	sys 0.011s
I20260709 12:16:58.473716 31468 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:58.474020 31776 raft_consensus.cc:359] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.474912 31776 raft_consensus.cc:385] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.475255 31776 raft_consensus.cc:740] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.478245 31776 consensus_queue.cc:260] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.483894 32095 raft_consensus.cc:2804] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:58.494217 32095 raft_consensus.cc:697] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:16:58.511557 31776 ts_tablet_manager.cc:1434] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 12:16:58.519831 32106 raft_consensus.cc:2804] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:58.516891 31470 leader_election.cc:343] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.521689 32098 raft_consensus.cc:2749] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:58.570348 31542 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
W20260709 12:16:58.579406 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 80136.4us
I20260709 12:16:58.583328 31774 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:58.584075 31774 ts_tablet_manager.cc:1403] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.528s	user 0.020s	sys 0.004s
I20260709 12:16:58.532964 31980 raft_consensus.cc:493] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 843512b2aa644b55a21b7e702943e02f)
I20260709 12:16:58.536077 32092 raft_consensus.cc:2749] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:58.585316 32095 consensus_queue.cc:237] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.602969 31647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.603755 31647 raft_consensus.cc:2468] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:58.593948 31776 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:58.536278 32106 raft_consensus.cc:697] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:58.596035 32126 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.598547 31772 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:58.609876 31772 ts_tablet_manager.cc:1403] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.463s	user 0.016s	sys 0.042s
I20260709 12:16:58.598626 32113 consensus_queue.cc:1048] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:16:58.598865 32100 raft_consensus.cc:2804] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:58.599164 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.622156 31774 raft_consensus.cc:359] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.623149 31774 raft_consensus.cc:385] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:58.624822 32122 log.cc:927] Time spent T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:16:58.622061 31693 leader_election.cc:304] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:58.633618 32106 consensus_queue.cc:237] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.621327 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.623705 31774 raft_consensus.cc:740] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.639585 31774 consensus_queue.cc:260] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.647936 31772 raft_consensus.cc:359] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.640921 31906 consensus_queue.cc:1048] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:58.649885 31772 raft_consensus.cc:385] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.650240 31772 raft_consensus.cc:740] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.606593 31980 raft_consensus.cc:515] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.655117 32041 raft_consensus.cc:2804] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.655577 32041 raft_consensus.cc:493] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.655941 32041 raft_consensus.cc:3060] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.656618 31774 ts_tablet_manager.cc:1434] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.004s
I20260709 12:16:58.651031 31772 consensus_queue.cc:260] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.618979 32100 raft_consensus.cc:697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:16:58.644793 31397 catalog_manager.cc:5697] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 2, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 2 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.666324 31772 ts_tablet_manager.cc:1434] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.056s	user 0.004s	sys 0.002s
I20260709 12:16:58.675904 31774 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:58.613284 32126 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.681159 31772 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
W20260709 12:16:58.664757 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 199063us
I20260709 12:16:58.692749 31500 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
W20260709 12:16:58.695535 31691 leader_election.cc:343] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:58.707897 32126 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:58.718004 31980 leader_election.cc:290] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:58.718575 31771 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:58.719094 31771 ts_tablet_manager.cc:1403] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.374s	user 0.011s	sys 0.013s
I20260709 12:16:58.692029 32100 consensus_queue.cc:237] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.725090 32091 leader_election.cc:290] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:58.729770 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2cd62a9ae5d4430bdb3799b63ca2137" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:58.735314 31776 tablet_bootstrap.cc:654] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.738744 32085 raft_consensus.cc:493] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.739289 32085 raft_consensus.cc:515] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.741302 31771 raft_consensus.cc:359] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.742056 31771 raft_consensus.cc:385] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.742337 31771 raft_consensus.cc:740] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.741176 31467 leader_election.cc:304] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:58.743039 31771 consensus_queue.cc:260] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.743790 32033 raft_consensus.cc:2749] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:58.746937 31401 catalog_manager.cc:5697] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.750252 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.752303 32085 leader_election.cc:290] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:58.759616 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.758226 32041 raft_consensus.cc:515] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.765625 31500 raft_consensus.cc:2393] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:58.782605 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d7a21102b64e7f95625a42c84e9ad0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.783380 31579 raft_consensus.cc:2468] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:58.806489 31691 leader_election.cc:304] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:58.807572 32085 raft_consensus.cc:2804] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:58.807968 32085 raft_consensus.cc:493] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:58.808315 32085 raft_consensus.cc:3060] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:58.811291 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 60737.6us
I20260709 12:16:58.814831 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
W20260709 12:16:58.817164 31691 leader_election.cc:343] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:58.817407 31616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 46454us
I20260709 12:16:58.818830 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:58.819836 31771 ts_tablet_manager.cc:1434] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.000s
I20260709 12:16:58.820149 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:58.820868 31771 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:58.826640 32087 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 843512b2aa644b55a21b7e702943e02f)
I20260709 12:16:58.827243 32087 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.827656 32041 leader_election.cc:290] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.828658 31774 tablet_bootstrap.cc:654] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.836072 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.836766 31577 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:16:58.834612 32087 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.837971 31691 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:58.838847 32087 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:58.839190 32087 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 843512b2aa644b55a21b7e702943e02f)
I20260709 12:16:58.839522 32087 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.846693 31772 tablet_bootstrap.cc:654] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.854462 31653 raft_consensus.cc:1275] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:58.856739 31813 consensus_queue.cc:1048] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:58.862658 31650 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.864929 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:58.865869 31653 raft_consensus.cc:3060] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.869714 31771 tablet_bootstrap.cc:654] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:58.870112 31500 raft_consensus.cc:1275] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:58.875087 31616 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:58.876014 31616 leader_election.cc:304] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:58.879599 32128 raft_consensus.cc:2749] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:58.888109 31980 raft_consensus.cc:493] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:58.890986 31980 raft_consensus.cc:515] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:58.893150 31980 leader_election.cc:290] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:58.905154 32038 consensus_queue.cc:1048] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:58.906872 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:58.907737 31649 raft_consensus.cc:2393] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:58.907323 32085 raft_consensus.cc:515] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:58.910636 31499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.911783 31499 raft_consensus.cc:2393] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:58.877280 31813 consensus_queue.cc:1048] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 12:16:58.920701 32087 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:58.922964 32087 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:58.923359 32085 leader_election.cc:290] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:58.924801 31576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:58.929361 32114 raft_consensus.cc:2749] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:58.930336 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d7a21102b64e7f95625a42c84e9ad0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:58.931160 31574 raft_consensus.cc:3060] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.933197 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d7a21102b64e7f95625a42c84e9ad0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:58.933867 31502 raft_consensus.cc:2468] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:58.936990 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9da10e70d33e4495babdca0057997818" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:58.955639 31400 catalog_manager.cc:5697] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:58.972002 31776 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:58.972574 31776 ts_tablet_manager.cc:1403] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.441s	user 0.019s	sys 0.042s
I20260709 12:16:58.957841 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d7a21102b64e7f95625a42c84e9ad0" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:58.977851 31776 raft_consensus.cc:359] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.978585 31776 raft_consensus.cc:385] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:58.978858 31776 raft_consensus.cc:740] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:58.979578 31776 consensus_queue.cc:260] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:58.955413 31617 leader_election.cc:304] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:58.982872 31502 raft_consensus.cc:3060] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:58.973239 32087 consensus_queue.cc:1048] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:16:58.995074 31771 tablet_bootstrap.cc:492] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:58.990464 31776 ts_tablet_manager.cc:1434] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
W20260709 12:16:59.001551 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 49961.3us
I20260709 12:16:59.002084 31980 raft_consensus.cc:3060] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.003826 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:59.004645 31649 raft_consensus.cc:2393] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:16:59.000967 31771 ts_tablet_manager.cc:1403] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.180s	user 0.015s	sys 0.026s
I20260709 12:16:59.000845 31653 raft_consensus.cc:2468] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:59.012845 31776 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:59.014804 31897 consensus_queue.cc:1048] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:59.014366 31576 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:16:59.031607 31574 raft_consensus.cc:2468] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:59.019558 32091 raft_consensus.cc:493] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.035641 31502 raft_consensus.cc:2468] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:59.037858 31771 raft_consensus.cc:359] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.032859 32091 raft_consensus.cc:515] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.045802 31776 tablet_bootstrap.cc:654] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.041369 31693 leader_election.cc:304] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 843512b2aa644b55a21b7e702943e02f
I20260709 12:16:59.042161 32120 consensus_queue.cc:1048] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:59.043762 31691 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:59.047230 32091 leader_election.cc:290] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:59.048182 31774 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:59.048722 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:59.052923 31467 leader_election.cc:343] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.055332 31774 ts_tablet_manager.cc:1403] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.396s	user 0.017s	sys 0.008s
I20260709 12:16:59.056412 31906 consensus_queue.cc:1048] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:59.057950 32087 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:59.058501 32087 raft_consensus.cc:697] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:59.058343 31774 raft_consensus.cc:359] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.059487 31690 leader_election.cc:304] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:59.060429 32120 raft_consensus.cc:2804] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:59.059700 32087 consensus_queue.cc:237] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.060891 32120 raft_consensus.cc:697] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:59.061781 32120 consensus_queue.cc:237] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:59.039187 31771 raft_consensus.cc:385] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.100587 31771 raft_consensus.cc:740] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.101418 31771 consensus_queue.cc:260] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.103077 31771 ts_tablet_manager.cc:1434] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.000s
I20260709 12:16:59.104197 31771 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:59.106487 31772 tablet_bootstrap.cc:492] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:59.107023 31772 ts_tablet_manager.cc:1403] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.426s	user 0.009s	sys 0.017s
I20260709 12:16:59.059607 31774 raft_consensus.cc:385] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.048906 31980 raft_consensus.cc:2749] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:59.112839 31774 raft_consensus.cc:740] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.117209 31774 consensus_queue.cc:260] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.116961 31932 raft_consensus.cc:2804] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:59.122028 31772 raft_consensus.cc:359] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.122787 31772 raft_consensus.cc:385] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.123060 31772 raft_consensus.cc:740] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.127097 31772 consensus_queue.cc:260] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.126628 31397 catalog_manager.cc:5697] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.136127 31397 catalog_manager.cc:5697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.118321 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:59.141148 31772 ts_tablet_manager.cc:1434] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.004s
I20260709 12:16:59.150022 31772 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
W20260709 12:16:59.126400 31693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 85114us
I20260709 12:16:59.151319 31932 raft_consensus.cc:697] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:59.160539 31774 ts_tablet_manager.cc:1434] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.105s	user 0.005s	sys 0.002s
I20260709 12:16:59.166692 31774 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:59.177974 31771 tablet_bootstrap.cc:654] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.188169 31980 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.191433 31774 tablet_bootstrap.cc:654] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:59.164655 31470 leader_election.cc:343] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.203183 31470 leader_election.cc:304] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:59.188939 31980 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.211659 31897 raft_consensus.cc:493] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.212903 32096 raft_consensus.cc:2749] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:59.227221 31772 tablet_bootstrap.cc:654] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.227809 31897 raft_consensus.cc:515] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.230034 31897 leader_election.cc:290] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
W20260709 12:16:59.219350 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 55969.8us
I20260709 12:16:59.231341 31401 catalog_manager.cc:5697] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.244984 31846 raft_consensus.cc:493] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.245582 31846 raft_consensus.cc:515] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.247741 31846 leader_election.cc:290] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:59.255316 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:59.256094 31501 raft_consensus.cc:2468] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:59.262997 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:59.263839 31650 raft_consensus.cc:2393] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:16:59.204841 31932 consensus_queue.cc:237] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.283685 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e302635e586c48819cf549549c5f4446" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:59.285785 31690 leader_election.cc:304] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:16:59.285859 31980 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:59.288589 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:59.292356 31832 consensus_queue.cc:1048] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:59.293654 32086 raft_consensus.cc:2804] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:59.294124 32086 raft_consensus.cc:493] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:59.294482 32086 raft_consensus.cc:3060] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.296813 31648 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.300459 32128 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.303783 31545 leader_election.cc:304] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
W20260709 12:16:59.308338 31470 consensus_peers.cc:597] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:59.319312 31846 raft_consensus.cc:3060] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.341037 31771 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:59.341944 31771 ts_tablet_manager.cc:1403] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.238s	user 0.013s	sys 0.005s
I20260709 12:16:59.312700 32128 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.352805 32128 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:59.354238 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:59.355234 31574 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:16:59.357053 31617 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:16:59.378393 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:16:59.379563 31723 raft_consensus.cc:2393] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:16:59.383273 32135 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:59.377099 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:59.409047 31616 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:59.409804 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:59.413512 31617 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.194:39965, user_credentials={real_user=slave}} blocked reactor thread for 56585.1us
I20260709 12:16:59.432458 31980 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:16:59.433043 31980 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:59.433622 31980 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:59.463547 31906 consensus_queue.cc:1048] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:59.465881 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:59.468091 31576 raft_consensus.cc:1275] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:59.475473 32077 raft_consensus.cc:2749] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:59.452970 31771 raft_consensus.cc:359] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.485105 31771 raft_consensus.cc:385] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.486171 31771 raft_consensus.cc:740] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.487855 31771 consensus_queue.cc:260] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.490514 31649 raft_consensus.cc:2468] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:16:59.515606 31771 ts_tablet_manager.cc:1434] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.173s	user 0.008s	sys 0.000s
I20260709 12:16:59.514017 31846 raft_consensus.cc:2749] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:59.376952 32086 raft_consensus.cc:515] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.532223 31776 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:59.532938 31776 ts_tablet_manager.cc:1403] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.520s	user 0.035s	sys 0.050s
I20260709 12:16:59.536870 31776 raft_consensus.cc:359] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:59.537573 31776 raft_consensus.cc:385] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.537948 31776 raft_consensus.cc:740] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.538641 31776 consensus_queue.cc:260] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:59.527985 31932 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:16:59.541324 31776 ts_tablet_manager.cc:1434] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:59.542443 31776 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:59.532919 32086 leader_election.cc:290] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:59.546262 31772 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:59.546819 31772 ts_tablet_manager.cc:1403] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.397s	user 0.075s	sys 0.117s
I20260709 12:16:59.546512 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:16:59.547499 31503 raft_consensus.cc:3060] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.548873 31776 tablet_bootstrap.cc:654] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.551014 31772 raft_consensus.cc:359] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.551749 31772 raft_consensus.cc:385] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.552040 31772 raft_consensus.cc:740] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.553028 31771 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:16:59.553157 31772 consensus_queue.cc:260] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.558477 31774 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:59.559092 31774 ts_tablet_manager.cc:1403] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.394s	user 0.016s	sys 0.003s
I20260709 12:16:59.566273 31772 ts_tablet_manager.cc:1434] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:16:59.583503 31813 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.585832 31813 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.587863 31813 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:59.590384 32161 consensus_queue.cc:1048] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:59.559362 32135 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:16:59.598490 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76fc75555e64cd3a5762910676fb4e3" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:16:59.599133 31650 raft_consensus.cc:3060] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.600970 31771 tablet_bootstrap.cc:654] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.612404 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:59.601568 31401 catalog_manager.cc:5697] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.617311 31401 catalog_manager.cc:5697] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:59.564589 31980 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:16:59.619736 31401 catalog_manager.cc:5697] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.622224 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:16:59.624405 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:16:59.626298 31503 raft_consensus.cc:2468] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
W20260709 12:16:59.628932 31542 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.633638 31832 raft_consensus.cc:493] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.636235 31832 raft_consensus.cc:515] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:16:59.638540 31980 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:16:59.639272 31832 leader_election.cc:290] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:16:59.639792 31774 raft_consensus.cc:359] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.640691 31774 raft_consensus.cc:385] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.641016 31774 raft_consensus.cc:740] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.650944 31772 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:16:59.658035 31774 consensus_queue.cc:260] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.665650 31776 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:16:59.666407 31776 ts_tablet_manager.cc:1403] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.124s	user 0.021s	sys 0.012s
I20260709 12:16:59.670447 31650 raft_consensus.cc:2468] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:16:59.670717 31771 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:59.671221 31771 ts_tablet_manager.cc:1403] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.144s	user 0.013s	sys 0.010s
W20260709 12:16:59.673243 32166 log.cc:927] Time spent T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: Append to log took a long time: real 0.079s	user 0.003s	sys 0.000s
I20260709 12:16:59.692503 31776 raft_consensus.cc:359] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.693190 31776 raft_consensus.cc:385] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.693404 31776 raft_consensus.cc:740] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.689327 31771 raft_consensus.cc:359] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.693809 31771 raft_consensus.cc:385] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.694070 31771 raft_consensus.cc:740] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:16:59.696421 31776 consensus_queue.cc:260] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.687042 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:16:59.698086 31771 consensus_queue.cc:260] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.690135 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:16:59.703326 31467 leader_election.cc:343] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.703802 31771 ts_tablet_manager.cc:1434] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:16:59.706983 31771 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
W20260709 12:16:59.712368 31545 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.713052 31771 tablet_bootstrap.cc:654] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.728588 31774 ts_tablet_manager.cc:1434] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.169s	user 0.007s	sys 0.001s
I20260709 12:16:59.730482 31774 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:16:59.731832 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:16:59.739145 31774 tablet_bootstrap.cc:654] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.740272 31772 tablet_bootstrap.cc:654] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.688097 31690 leader_election.cc:304] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
W20260709 12:16:59.741930 31690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 62097.2us
I20260709 12:16:59.742918 32085 raft_consensus.cc:2804] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:59.743436 32085 raft_consensus.cc:697] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:16:59.744252 32085 consensus_queue.cc:237] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.745594 31897 consensus_queue.cc:1048] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:59.697748 32092 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.748286 32092 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.750510 32092 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:16:59.750814 31616 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
W20260709 12:16:59.751809 32172 log.cc:927] Time spent T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:16:59.752326 31980 raft_consensus.cc:2749] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:16:59.713016 31545 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:59.758414 31906 raft_consensus.cc:2749] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:59.759598 31545 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 47277.1us
I20260709 12:16:59.784039 31776 ts_tablet_manager.cc:1434] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.117s	user 0.013s	sys 0.003s
I20260709 12:16:59.800203 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
W20260709 12:16:59.808758 31468 leader_election.cc:343] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.809384 31468 leader_election.cc:304] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:16:59.811048 32091 raft_consensus.cc:2749] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:59.854245 31690 leader_election.cc:343] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.854897 31776 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:16:59.865657 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:16:59.879456 31693 leader_election.cc:343] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.879954 31693 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:16:59.885646 32087 raft_consensus.cc:2749] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:59.886411 31651 raft_consensus.cc:1240] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Rejecting Update request from peer bf0f149240cc4f27b97810c90eba7ff9 for earlier term 2. Current term is 3. Ops: []
I20260709 12:16:59.893868 32161 consensus_queue.cc:1059] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:59.843786 31470 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 137598us
I20260709 12:16:59.906193 31648 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.913349 31832 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:59.886384 32087 consensus_queue.cc:1048] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 12:16:59.944720 31776 tablet_bootstrap.cc:654] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:59.933471 32087 consensus_queue.cc:1048] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:16:59.920732 31832 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:16:59.952090 31832 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:16:59.955034 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:16:59.956607 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
W20260709 12:16:59.956887 31470 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:59.957309 31575 raft_consensus.cc:2468] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:16:59.962421 31468 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:16:59.963742 32176 raft_consensus.cc:2804] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:59.964144 32176 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:59.964462 32176 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:59.973016 31771 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:16:59.973603 31771 ts_tablet_manager.cc:1403] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.267s	user 0.020s	sys 0.001s
I20260709 12:16:59.978104 31772 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:16:59.978672 31772 ts_tablet_manager.cc:1403] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.328s	user 0.018s	sys 0.000s
I20260709 12:16:59.984514 31774 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:16:59.992264 31771 raft_consensus.cc:359] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:16:59.993034 31771 raft_consensus.cc:385] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:59.985061 31774 ts_tablet_manager.cc:1403] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.255s	user 0.040s	sys 0.038s
I20260709 12:16:59.993287 31771 raft_consensus.cc:740] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.001734 31771 consensus_queue.cc:260] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.005031 31771 ts_tablet_manager.cc:1434] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:17:00.006628 32176 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.008368 31771 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.011615 32114 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:00.012187 32114 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.014017 32176 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.015056 31776 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.015156 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:00.015939 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:00.017803 31575 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:00.017971 31470 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.018281 31771 tablet_bootstrap.cc:654] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.021786 31776 ts_tablet_manager.cc:1403] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.167s	user 0.038s	sys 0.014s
I20260709 12:17:00.023252 31771 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.023754 31771 ts_tablet_manager.cc:1403] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:17:00.043747 31771 raft_consensus.cc:359] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.044528 31771 raft_consensus.cc:385] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.044812 31771 raft_consensus.cc:740] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.045047 31776 raft_consensus.cc:359] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.045535 31771 consensus_queue.cc:260] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.046056 31776 raft_consensus.cc:385] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.047741 31771 ts_tablet_manager.cc:1434] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.007s
I20260709 12:17:00.048884 31771 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.052393 31772 raft_consensus.cc:359] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.053094 31772 raft_consensus.cc:385] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.053380 31772 raft_consensus.cc:740] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.054018 31772 consensus_queue.cc:260] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.009810 31932 raft_consensus.cc:3055] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:00.055214 31932 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:00.055858 31771 tablet_bootstrap.cc:654] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.056244 31932 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.056749 32114 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:00.046595 31776 raft_consensus.cc:740] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.067800 31653 raft_consensus.cc:1240] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Rejecting Update request from peer bf0f149240cc4f27b97810c90eba7ff9 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:17:00.069025 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.074683 31772 ts_tablet_manager.cc:1434] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 12:17:00.079056 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.080628 31813 raft_consensus.cc:493] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.081314 31813 raft_consensus.cc:515] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.082039 31616 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:00.082844 32077 raft_consensus.cc:2749] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:00.083716 31813 leader_election.cc:290] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.085628 31397 catalog_manager.cc:5697] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.072063 31932 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:00.072271 31776 consensus_queue.cc:260] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.088955 31774 raft_consensus.cc:359] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.089692 31774 raft_consensus.cc:385] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.089962 31774 raft_consensus.cc:740] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.090559 31774 consensus_queue.cc:260] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.095952 31575 raft_consensus.cc:2468] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:17:00.103976 31468 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:00.105445 32177 raft_consensus.cc:2804] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.105939 32177 raft_consensus.cc:697] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:00.106750 32177 consensus_queue.cc:237] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.108647 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.111140 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.111809 31723 raft_consensus.cc:2468] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
W20260709 12:17:00.116238 31542 leader_election.cc:343] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.126438 31542 leader_election.cc:304] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:00.130733 31906 raft_consensus.cc:2804] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.131121 31906 raft_consensus.cc:493] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.131428 31906 raft_consensus.cc:3060] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.133301 31776 ts_tablet_manager.cc:1434] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.111s	user 0.007s	sys 0.001s
I20260709 12:17:00.138708 31772 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.140789 31776 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.148458 31774 ts_tablet_manager.cc:1434] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.153s	user 0.006s	sys 0.000s
I20260709 12:17:00.152391 31776 tablet_bootstrap.cc:654] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.153604 31774 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.155885 31906 raft_consensus.cc:515] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.164170 31906 leader_election.cc:290] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.166360 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:00.166785 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:00.167439 31722 raft_consensus.cc:3060] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:00.168437 31542 leader_election.cc:343] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.209836 31772 tablet_bootstrap.cc:654] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.211138 31774 tablet_bootstrap.cc:654] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.220692 32092 consensus_queue.cc:1048] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:00.235039 31722 raft_consensus.cc:2468] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:00.240144 31771 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.240645 31771 ts_tablet_manager.cc:1403] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.192s	user 0.016s	sys 0.018s
I20260709 12:17:00.247488 32077 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.244880 31542 leader_election.cc:304] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:00.249502 32077 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.249555 31400 catalog_manager.cc:5697] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.251825 32077 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:00.252887 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.247617 31776 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.254633 31776 ts_tablet_manager.cc:1403] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.114s	user 0.032s	sys 0.025s
I20260709 12:17:00.247656 31771 raft_consensus.cc:359] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.255158 31771 raft_consensus.cc:385] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.255673 31771 raft_consensus.cc:740] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.257156 31906 raft_consensus.cc:2804] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.260246 31906 raft_consensus.cc:697] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:00.261127 31906 consensus_queue.cc:237] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.256842 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.262691 31578 raft_consensus.cc:2393] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in this term.
I20260709 12:17:00.258025 31776 raft_consensus.cc:359] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.264012 31776 raft_consensus.cc:385] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.264299 31776 raft_consensus.cc:740] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.259055 31771 consensus_queue.cc:260] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.265021 31776 consensus_queue.cc:260] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.267102 31776 ts_tablet_manager.cc:1434] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:17:00.268159 31776 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.269352 31617 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:00.270773 32114 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.273669 32092 consensus_queue.cc:1048] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:00.279888 32114 raft_consensus.cc:2749] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:00.285993 31771 ts_tablet_manager.cc:1434] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.002s
I20260709 12:17:00.287185 31771 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.295648 31776 tablet_bootstrap.cc:654] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.299364 32114 raft_consensus.cc:493] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.299909 32114 raft_consensus.cc:515] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.302042 32114 leader_election.cc:290] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.303573 31772 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:00.304076 31772 ts_tablet_manager.cc:1403] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.166s	user 0.009s	sys 0.009s
I20260709 12:17:00.304142 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.307204 31651 raft_consensus.cc:3060] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.307056 31772 raft_consensus.cc:359] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.307755 31772 raft_consensus.cc:385] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.308038 31772 raft_consensus.cc:740] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.315445 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "098e97adee224f86ba1bfa07ce392b68" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.317948 31616 leader_election.cc:304] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:00.326633 31772 consensus_queue.cc:260] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.329541 31774 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:00.329679 31772 ts_tablet_manager.cc:1434] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 12:17:00.330231 31774 ts_tablet_manager.cc:1403] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.009s
I20260709 12:17:00.330821 31772 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.337729 31772 tablet_bootstrap.cc:654] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.341404 31980 raft_consensus.cc:2749] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:00.316723 31397 catalog_manager.cc:5697] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:00.352504 31832 raft_consensus.cc:493] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.352983 31832 raft_consensus.cc:515] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.355741 31776 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.356180 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.356789 31776 ts_tablet_manager.cc:1403] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.024s
W20260709 12:17:00.358217 31467 leader_election.cc:343] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.358800 31832 leader_election.cc:290] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.359846 31776 raft_consensus.cc:359] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.360551 31776 raft_consensus.cc:385] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.360822 31776 raft_consensus.cc:740] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.361703 31776 consensus_queue.cc:260] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.365960 31776 ts_tablet_manager.cc:1434] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:17:00.367730 31776 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.368489 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:17:00.373179 31470 leader_election.cc:343] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.373631 31470 leader_election.cc:304] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:00.374502 32132 raft_consensus.cc:2749] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:00.378091 31776 tablet_bootstrap.cc:654] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.333048 31774 raft_consensus.cc:359] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.378912 31774 raft_consensus.cc:385] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.379238 31774 raft_consensus.cc:740] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.379937 31774 consensus_queue.cc:260] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.382473 31774 ts_tablet_manager.cc:1434] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 12:17:00.383412 31774 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.385828 31771 tablet_bootstrap.cc:654] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.398567 31774 tablet_bootstrap.cc:654] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.404726 32205 raft_consensus.cc:493] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.405280 32205 raft_consensus.cc:515] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.407486 32205 leader_election.cc:290] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.408540 31498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
W20260709 12:17:00.410676 31542 leader_election.cc:343] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.412722 31771 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.413213 31771 ts_tablet_manager.cc:1403] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.006s
I20260709 12:17:00.413544 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:00.414176 31651 raft_consensus.cc:2468] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:17:00.415714 31545 leader_election.cc:304] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:00.416437 31771 raft_consensus.cc:359] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.417128 31771 raft_consensus.cc:385] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.417395 31771 raft_consensus.cc:740] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.417725 31772 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:00.418215 31772 ts_tablet_manager.cc:1403] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.000s
I20260709 12:17:00.418224 31771 consensus_queue.cc:260] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.420146 32135 raft_consensus.cc:2804] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.420521 32135 raft_consensus.cc:493] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.420847 32135 raft_consensus.cc:3060] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.424729 31771 ts_tablet_manager.cc:1434] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:17:00.420989 31772 raft_consensus.cc:359] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.425911 31771 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.426066 31772 raft_consensus.cc:385] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.428589 31772 raft_consensus.cc:740] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.429417 31772 consensus_queue.cc:260] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.431494 31772 ts_tablet_manager.cc:1434] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:17:00.432255 31771 tablet_bootstrap.cc:654] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.436735 31774 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:00.436735 31776 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.437297 31774 ts_tablet_manager.cc:1403] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 12:17:00.437367 31776 ts_tablet_manager.cc:1403] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.070s	user 0.013s	sys 0.000s
I20260709 12:17:00.439976 31776 raft_consensus.cc:359] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.440603 31776 raft_consensus.cc:385] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.440877 31776 raft_consensus.cc:740] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.441059 31774 raft_consensus.cc:359] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.441709 31776 consensus_queue.cc:260] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.443989 32135 raft_consensus.cc:515] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.444469 31776 ts_tablet_manager.cc:1434] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:17:00.446306 32135 leader_election.cc:290] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.447652 31776 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.441766 31774 raft_consensus.cc:385] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.450446 31774 raft_consensus.cc:740] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.451462 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:00.451409 31774 consensus_queue.cc:260] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.452082 31649 raft_consensus.cc:3060] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.458006 31774 ts_tablet_manager.cc:1434] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 12:17:00.458688 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:00.459136 31774 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.461040 31649 raft_consensus.cc:2468] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
W20260709 12:17:00.465752 31542 leader_election.cc:343] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.467254 31545 leader_election.cc:304] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:00.468073 32205 raft_consensus.cc:2804] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.468600 32205 raft_consensus.cc:697] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:00.469514 32205 consensus_queue.cc:237] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:17:00.475324 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:00.481822 31772 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.483932 31774 tablet_bootstrap.cc:654] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.490811 31776 tablet_bootstrap.cc:654] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.494454 31772 tablet_bootstrap.cc:654] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.498245 31906 raft_consensus.cc:493] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.504488 31906 raft_consensus.cc:515] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.500435 31771 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.505972 31771 ts_tablet_manager.cc:1403] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.080s	user 0.007s	sys 0.008s
I20260709 12:17:00.506450 31776 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.507066 31776 ts_tablet_manager.cc:1403] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.004s
I20260709 12:17:00.500310 31401 catalog_manager.cc:5697] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:00.507455 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.508173 31500 raft_consensus.cc:2468] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:17:00.508762 31771 raft_consensus.cc:359] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.509470 31771 raft_consensus.cc:385] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.509728 31771 raft_consensus.cc:740] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.509763 31542 leader_election.cc:304] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:00.509939 31776 raft_consensus.cc:359] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.510620 31776 raft_consensus.cc:385] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.510704 32205 raft_consensus.cc:2804] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.510484 31771 consensus_queue.cc:260] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.511014 31776 raft_consensus.cc:740] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.511149 32205 raft_consensus.cc:493] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.511739 32205 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.502564 31774 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:00.512542 31774 ts_tablet_manager.cc:1403] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.004s
I20260709 12:17:00.506484 31906 leader_election.cc:290] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.515369 31774 raft_consensus.cc:359] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.516001 31774 raft_consensus.cc:385] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.516250 31774 raft_consensus.cc:740] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.517033 31774 consensus_queue.cc:260] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.517215 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
W20260709 12:17:00.519260 31545 leader_election.cc:343] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.519985 31774 ts_tablet_manager.cc:1434] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:00.521019 31774 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.522673 32205 raft_consensus.cc:515] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.511991 31776 consensus_queue.cc:260] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.522185 31771 ts_tablet_manager.cc:1434] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:17:00.528102 31771 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.528888 31774 tablet_bootstrap.cc:654] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.531301 31776 ts_tablet_manager.cc:1434] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:17:00.533304 31772 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:00.533824 31772 ts_tablet_manager.cc:1403] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.000s
I20260709 12:17:00.534785 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
W20260709 12:17:00.536185 31545 leader_election.cc:343] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.536835 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:00.537483 31776 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.537483 31500 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.541229 31832 raft_consensus.cc:493] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.541772 31832 raft_consensus.cc:515] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.544049 31832 leader_election.cc:290] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:00.544731 31771 tablet_bootstrap.cc:654] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.545198 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.545373 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53b9a430b2d54764b10115eae2e0e653" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.546011 31723 raft_consensus.cc:2393] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7dd13c0178c84f7aaab5c435a8b205e4 in current term 1: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:17:00.549049 31468 leader_election.cc:304] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:00.550006 31774 tablet_bootstrap.cc:492] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:00.550428 31832 raft_consensus.cc:3060] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.550889 32114 raft_consensus.cc:493] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.551649 31776 tablet_bootstrap.cc:654] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.551388 32114 raft_consensus.cc:515] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.552678 32132 consensus_queue.cc:1048] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:00.554154 31774 ts_tablet_manager.cc:1403] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.004s
I20260709 12:17:00.554796 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.536582 31772 raft_consensus.cc:359] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.557155 31772 raft_consensus.cc:385] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.557523 31772 raft_consensus.cc:740] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
W20260709 12:17:00.556612 31617 leader_election.cc:343] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.558346 31772 consensus_queue.cc:260] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.560814 31772 ts_tablet_manager.cc:1434] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.000s
I20260709 12:17:00.556802 32114 leader_election.cc:290] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.562124 31774 raft_consensus.cc:359] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.562892 31774 raft_consensus.cc:385] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.563148 31774 raft_consensus.cc:740] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.566262 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:17:00.567772 31616 leader_election.cc:343] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.568156 31616 leader_election.cc:304] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:00.569679 32114 raft_consensus.cc:2749] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:00.575160 32132 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.575714 32132 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.579733 32132 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.580300 32205 leader_election.cc:290] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.586160 31772 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.588047 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:00.589258 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:17:00.589676 31470 leader_election.cc:343] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.589926 31723 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 0.
I20260709 12:17:00.591265 31467 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:00.591941 32132 raft_consensus.cc:2804] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.594830 31774 consensus_queue.cc:260] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.605585 31500 raft_consensus.cc:2468] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:00.592347 32132 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.608448 31832 raft_consensus.cc:2749] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:00.608513 31542 leader_election.cc:304] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:00.612612 31771 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.612984 32205 raft_consensus.cc:2804] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.613205 31771 ts_tablet_manager.cc:1403] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.085s	user 0.006s	sys 0.015s
I20260709 12:17:00.613572 32205 raft_consensus.cc:697] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:00.614436 32205 consensus_queue.cc:237] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.616137 31771 raft_consensus.cc:359] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.616801 31771 raft_consensus.cc:385] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.617060 31771 raft_consensus.cc:740] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.617712 31771 consensus_queue.cc:260] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.619753 31771 ts_tablet_manager.cc:1434] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:00.620677 31771 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.607132 32132 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.610025 31774 ts_tablet_manager.cc:1434] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 12:17:00.626958 31772 tablet_bootstrap.cc:654] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.627727 31774 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.627836 32114 raft_consensus.cc:493] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.629449 32114 raft_consensus.cc:515] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.639271 31772 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:00.639796 31772 ts_tablet_manager.cc:1403] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.007s
I20260709 12:17:00.640074 31774 tablet_bootstrap.cc:654] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.642263 32096 consensus_queue.cc:1048] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:00.642674 31772 raft_consensus.cc:359] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.643347 31772 raft_consensus.cc:385] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.633972 31776 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.643616 31772 raft_consensus.cc:740] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.644066 31776 ts_tablet_manager.cc:1403] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.014s
I20260709 12:17:00.644457 31772 consensus_queue.cc:260] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.646796 31772 ts_tablet_manager.cc:1434] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:00.646854 31776 raft_consensus.cc:359] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.647612 31776 raft_consensus.cc:385] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.626494 31771 tablet_bootstrap.cc:654] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.647867 31772 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.654882 31772 tablet_bootstrap.cc:654] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.647887 31776 raft_consensus.cc:740] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.658694 31776 consensus_queue.cc:260] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.659430 32114 leader_election.cc:290] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.661114 31776 ts_tablet_manager.cc:1434] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:17:00.661195 32132 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.661787 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.662254 31776 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:00.663590 32132 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.662465 31574 raft_consensus.cc:2468] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:00.666301 31617 leader_election.cc:304] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:00.669842 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:00.667951 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:00.673815 31722 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.674784 31771 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.675277 31771 ts_tablet_manager.cc:1403] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.006s
I20260709 12:17:00.676563 31776 tablet_bootstrap.cc:654] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:00.671582 31470 leader_election.cc:343] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:00.683796 31771 raft_consensus.cc:359] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.684584 31771 raft_consensus.cc:385] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.684883 31771 raft_consensus.cc:740] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.689486 31771 consensus_queue.cc:260] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.690225 31932 raft_consensus.cc:493] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.690948 31932 raft_consensus.cc:515] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.693288 31771 ts_tablet_manager.cc:1434] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:17:00.694484 31771 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:00.694766 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.695067 31932 leader_election.cc:290] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.679292 32114 raft_consensus.cc:2804] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.695451 31723 raft_consensus.cc:2468] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:00.698356 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:00.721649 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:00.722901 31578 raft_consensus.cc:2468] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:00.724768 31691 leader_election.cc:304] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:00.705595 32114 raft_consensus.cc:493] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.731309 32114 raft_consensus.cc:3060] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.728037 31932 raft_consensus.cc:2804] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.731048 31722 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 1.
I20260709 12:17:00.745879 31932 raft_consensus.cc:493] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.751945 32120 raft_consensus.cc:493] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.752485 32120 raft_consensus.cc:515] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.754693 32120 leader_election.cc:290] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.755704 31776 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: No bootstrap required, opened a new log
I20260709 12:17:00.756263 31776 ts_tablet_manager.cc:1403] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.094s	user 0.013s	sys 0.016s
I20260709 12:17:00.758580 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.757899 31467 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:00.747409 31932 raft_consensus.cc:3060] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.762207 31653 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:00.762895 31653 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:00.763314 31772 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:00.764027 31772 ts_tablet_manager.cc:1403] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.116s	user 0.018s	sys 0.021s
I20260709 12:17:00.764828 31832 raft_consensus.cc:2804] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.765352 31832 raft_consensus.cc:697] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:00.766242 31832 consensus_queue.cc:237] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.767746 31772 raft_consensus.cc:359] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.770290 31772 raft_consensus.cc:385] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.770666 31772 raft_consensus.cc:740] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.771543 31772 consensus_queue.cc:260] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.771514 31504 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:00.775023 31690 leader_election.cc:304] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:00.776383 31932 raft_consensus.cc:515] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.777456 31906 consensus_queue.cc:1048] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:00.779302 32085 raft_consensus.cc:2804] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.781669 31772 ts_tablet_manager.cc:1434] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 12:17:00.779695 32085 raft_consensus.cc:493] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.782788 31772 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:00.784653 31771 tablet_bootstrap.cc:654] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.782739 32085 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.779775 32114 raft_consensus.cc:515] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.806638 31932 leader_election.cc:290] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:00.807861 32114 leader_election.cc:290] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:00.809305 31651 raft_consensus.cc:2393] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:00.811072 31772 tablet_bootstrap.cc:654] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:00.811898 32135 raft_consensus.cc:493] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:00.820724 31690 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.193:37911, user_credentials={real_user=slave}} blocked reactor thread for 45784us
I20260709 12:17:00.825316 31577 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:00.826005 31577 raft_consensus.cc:3060] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.819514 32205 consensus_queue.cc:1048] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:17:00.812405 32135 raft_consensus.cc:515] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.831357 32135 leader_election.cc:290] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.833262 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c8433c534b4d19bc10a5cb0ca094ba" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.834260 31723 raft_consensus.cc:2468] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:17:00.835371 31577 raft_consensus.cc:2468] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:00.836583 31617 leader_election.cc:304] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:00.836973 31542 leader_election.cc:304] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:00.837641 32114 raft_consensus.cc:2804] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:00.838240 32135 raft_consensus.cc:2804] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:00.838768 32135 raft_consensus.cc:493] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:00.842396 32114 raft_consensus.cc:697] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:00.842293 31498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c8433c534b4d19bc10a5cb0ca094ba" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.845729 31498 raft_consensus.cc:2468] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:17:00.844663 32114 consensus_queue.cc:237] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:00.847760 31774 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:00.839219 32135 raft_consensus.cc:3060] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:00.854367 31774 ts_tablet_manager.cc:1403] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.227s	user 0.024s	sys 0.009s
I20260709 12:17:00.848104 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:00.871558 31774 raft_consensus.cc:359] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.872519 31774 raft_consensus.cc:385] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.872871 31774 raft_consensus.cc:740] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.873749 31774 consensus_queue.cc:260] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.875586 31400 catalog_manager.cc:5697] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:00.877671 31813 raft_consensus.cc:493] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:00.880338 31813 raft_consensus.cc:515] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:00.882553 31813 leader_election.cc:290] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
W20260709 12:17:00.883632 31693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 125683us
I20260709 12:17:00.889899 31578 raft_consensus.cc:2393] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:00.897151 31774 ts_tablet_manager.cc:1434] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.002s
I20260709 12:17:00.917945 31776 raft_consensus.cc:359] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.918754 31776 raft_consensus.cc:385] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:00.919030 31776 raft_consensus.cc:740] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:00.920758 31776 consensus_queue.cc:260] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:00.935439 31774 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:00.966858 31776 ts_tablet_manager.cc:1434] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.210s	user 0.006s	sys 0.003s
I20260709 12:17:00.961688 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:00.972687 31502 raft_consensus.cc:2468] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
I20260709 12:17:00.973053 32085 raft_consensus.cc:515] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:00.976141 32085 leader_election.cc:290] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:00.962226 31723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:00.978783 31723 raft_consensus.cc:2393] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:00.980684 31771 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:00.981186 31771 ts_tablet_manager.cc:1403] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.287s	user 0.019s	sys 0.001s
I20260709 12:17:00.993139 32132 raft_consensus.cc:493] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:01.002115 31772 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:01.002653 31772 ts_tablet_manager.cc:1403] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.220s	user 0.035s	sys 0.000s
I20260709 12:17:01.006183 31774 tablet_bootstrap.cc:654] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:01.005280 31693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:38461, user_credentials={real_user=slave}} blocked reactor thread for 77120us
I20260709 12:17:00.970999 31724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:00.993973 32132 raft_consensus.cc:515] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:01.028048 31771 raft_consensus.cc:359] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.032634 31771 raft_consensus.cc:385] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.032946 31771 raft_consensus.cc:740] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.033680 31771 consensus_queue.cc:260] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.000922 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:01.053527 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85a31e6ff114d30b4ae3a4453403a01" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:01.047857 31502 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.037812 31724 raft_consensus.cc:2468] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 0.
W20260709 12:17:01.088361 31616 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 70246.5us
I20260709 12:17:01.110038 32135 raft_consensus.cc:515] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:01.170753 31771 ts_tablet_manager.cc:1434] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.189s	user 0.031s	sys 0.016s
I20260709 12:17:01.172875 31542 leader_election.cc:304] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:01.184829 31771 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:01.187781 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:01.188611 31651 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.199309 32135 leader_election.cc:290] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.236191 31651 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:01.204710 32243 raft_consensus.cc:493] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:17:01.215386 32230 log.cc:927] Time spent T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:17:01.217535 32241 raft_consensus.cc:2804] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:01.264542 31772 raft_consensus.cc:359] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.265331 31772 raft_consensus.cc:385] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.265681 31772 raft_consensus.cc:740] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.252939 32241 raft_consensus.cc:493] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:01.245101 32243 raft_consensus.cc:515] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.283354 31503 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c8433c534b4d19bc10a5cb0ca094ba" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:01.284571 31503 raft_consensus.cc:3060] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.285027 32241 raft_consensus.cc:3060] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.285912 31693 leader_election.cc:304] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:01.263727 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6c8433c534b4d19bc10a5cb0ca094ba" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:01.287226 32161 raft_consensus.cc:2749] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:01.287336 31693 leader_election.cc:304] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:01.273887 31498 raft_consensus.cc:3060] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.287523 31725 raft_consensus.cc:3060] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.272855 31502 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
W20260709 12:17:01.302625 31545 consensus_peers.cc:597] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:01.304811 31503 raft_consensus.cc:2468] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:01.305480 32161 raft_consensus.cc:2804] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:01.306011 32161 raft_consensus.cc:697] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:01.306865 32161 consensus_queue.cc:237] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.312292 31813 consensus_queue.cc:1048] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:01.315351 31772 consensus_queue.cc:260] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.344276 32086 raft_consensus.cc:493] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:01.360095 31772 ts_tablet_manager.cc:1434] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.357s	user 0.010s	sys 0.001s
W20260709 12:17:01.368144 31470 consensus_peers.cc:597] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:01.383422 31774 tablet_bootstrap.cc:492] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:01.384018 31774 ts_tablet_manager.cc:1403] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.461s	user 0.021s	sys 0.009s
I20260709 12:17:01.364012 32086 raft_consensus.cc:515] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.387645 32246 raft_consensus.cc:493] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:01.388273 32246 raft_consensus.cc:515] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.390389 32243 leader_election.cc:290] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.390782 32246 leader_election.cc:290] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:01.388584 32086 leader_election.cc:290] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.393241 32161 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:01.396373 32161 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.399170 32161 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:01.413673 31771 tablet_bootstrap.cc:654] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:01.414074 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:01.414806 31504 raft_consensus.cc:2468] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:01.413699 31772 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: Bootstrap starting.
I20260709 12:17:01.417642 31576 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:01.418293 31576 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:01.419323 32132 leader_election.cc:290] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:01.419584 31691 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:01.420698 32120 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:01.421056 32120 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:01.421381 32120 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:01.417177 31575 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbfef93226df40f3a8d07af88af622fb" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:01.424257 31648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:01.424589 31575 raft_consensus.cc:2468] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:01.440398 31616 leader_election.cc:304] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:01.437101 31774 raft_consensus.cc:359] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.447248 31774 raft_consensus.cc:385] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.447561 31774 raft_consensus.cc:740] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.435645 31502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:01.451457 31772 tablet_bootstrap.cc:654] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:01.452749 31542 leader_election.cc:304] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:01.454003 32247 raft_consensus.cc:2804] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:01.436295 31617 leader_election.cc:304] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:01.458495 32247 raft_consensus.cc:493] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:01.458914 32247 raft_consensus.cc:3060] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.459300 31980 raft_consensus.cc:2804] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:01.459733 31980 raft_consensus.cc:493] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:01.460081 31980 raft_consensus.cc:3060] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.462014 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c130f9abd9da4eaab759d4a5de2c1e86" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:01.462448 32254 consensus_queue.cc:1048] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:01.466456 31774 consensus_queue.cc:260] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.470604 31646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:01.471078 32254 raft_consensus.cc:2804] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:01.429523 31725 raft_consensus.cc:2468] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:01.461802 31402 catalog_manager.cc:5697] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:01.464324 31502 raft_consensus.cc:2468] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:01.429037 32077 consensus_queue.cc:1048] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 12:17:01.481190 31690 leader_election.cc:304] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:01.471649 32254 raft_consensus.cc:697] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:01.477838 31719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:01.486928 31468 leader_election.cc:304] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:01.492074 31774 ts_tablet_manager.cc:1434] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.108s	user 0.010s	sys 0.001s
I20260709 12:17:01.486295 32224 raft_consensus.cc:2804] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:01.484059 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbfef93226df40f3a8d07af88af622fb" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:01.501986 31721 raft_consensus.cc:2468] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:01.505155 31719 raft_consensus.cc:2468] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 0.
I20260709 12:17:01.505853 32120 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.506626 32224 raft_consensus.cc:493] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:01.507053 32224 raft_consensus.cc:3060] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.508247 32120 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 4 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:01.491464 32176 raft_consensus.cc:2749] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:01.513541 31774 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: Bootstrap starting.
I20260709 12:17:01.496708 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:01.489678 32236 consensus_queue.cc:1048] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:17:01.537401 31401 catalog_manager.cc:5697] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:01.544556 31450 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:01.532697 31650 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 3.
I20260709 12:17:01.560427 32248 consensus_queue.cc:1048] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:17:01.572167 32224 raft_consensus.cc:515] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.579326 31980 raft_consensus.cc:515] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.580307 31771 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:01.548023 32241 raft_consensus.cc:515] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:01.526791 32254 consensus_queue.cc:237] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:01.586440 31648 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:01.590636 31771 ts_tablet_manager.cc:1403] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.408s	user 0.013s	sys 0.009s
I20260709 12:17:01.596150 31774 tablet_bootstrap.cc:654] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:01.599665 31648 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:01.601645 32077 consensus_queue.cc:1048] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:01.603525 31980 leader_election.cc:290] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:01.624430 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbfef93226df40f3a8d07af88af622fb" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:01.630376 31574 raft_consensus.cc:3060] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.611655 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:01.641695 31578 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:17:01.670509 32241 leader_election.cc:290] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.674069 31725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbfef93226df40f3a8d07af88af622fb" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:01.674769 31725 raft_consensus.cc:3060] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.676041 31574 raft_consensus.cc:2468] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:01.677822 31726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:01.678107 31617 leader_election.cc:304] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:01.678480 31726 raft_consensus.cc:3060] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.680374 31501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81a98a7e3a4c46e9a01e39bb9924afc0" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:01.681255 31501 raft_consensus.cc:3060] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.693104 31725 raft_consensus.cc:2468] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:01.694635 31771 raft_consensus.cc:359] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:01.695369 31771 raft_consensus.cc:385] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.695621 31771 raft_consensus.cc:740] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.694651 31726 raft_consensus.cc:2468] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:01.696386 31771 consensus_queue.cc:260] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:01.697062 31501 raft_consensus.cc:2468] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:01.698382 32077 raft_consensus.cc:2804] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:01.698909 32077 raft_consensus.cc:697] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:01.699720 32077 consensus_queue.cc:237] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.725968 32247 raft_consensus.cc:515] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.727269 31646 raft_consensus.cc:2393] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:01.728683 32247 leader_election.cc:290] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.738576 31771 ts_tablet_manager.cc:1434] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.143s	user 0.007s	sys 0.000s
I20260709 12:17:01.739802 31771 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: Bootstrap starting.
I20260709 12:17:01.740556 32224 leader_election.cc:290] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:01.742211 31648 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 4.
I20260709 12:17:01.742998 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:01.743427 31495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:01.744104 31721 raft_consensus.cc:2393] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 1: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:01.744129 31495 raft_consensus.cc:3060] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.749521 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:01.750381 31650 raft_consensus.cc:2393] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 1: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:01.750567 31504 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f6baf18d4c74681ab192824db201eec" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:01.752390 31646 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:01.756000 31693 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:01.757485 31578 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 4.
I20260709 12:17:01.757579 32234 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:01.759119 32234 raft_consensus.cc:697] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:01.760905 31616 leader_election.cc:304] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:01.756397 31542 leader_election.cc:304] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:01.761998 32234 consensus_queue.cc:237] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.764474 32135 raft_consensus.cc:2804] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:01.765054 32135 raft_consensus.cc:697] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:01.766304 32135 consensus_queue.cc:237] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:01.790446 31771 tablet_bootstrap.cc:654] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:01.773120 32246 raft_consensus.cc:2749] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:01.772817 32120 consensus_queue.cc:1048] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260709 12:17:01.901599 32239 consensus_queue.cc:1048] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:01.918988 31772 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: No bootstrap required, opened a new log
I20260709 12:17:01.919590 31772 ts_tablet_manager.cc:1403] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: Time spent bootstrapping tablet: real 0.550s	user 0.017s	sys 0.026s
I20260709 12:17:01.933698 31772 raft_consensus.cc:359] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.936384 31772 raft_consensus.cc:385] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.936669 31772 raft_consensus.cc:740] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.937361 31772 consensus_queue.cc:260] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:01.942951 31495 raft_consensus.cc:2468] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:01.947626 31690 leader_election.cc:304] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:01.949018 31774 tablet_bootstrap.cc:492] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: No bootstrap required, opened a new log
I20260709 12:17:01.947642 31772 ts_tablet_manager.cc:1434] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:17:01.950354 31774 ts_tablet_manager.cc:1403] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: Time spent bootstrapping tablet: real 0.452s	user 0.024s	sys 0.019s
I20260709 12:17:01.955456 32224 raft_consensus.cc:2804] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:01.975092 31771 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: No bootstrap required, opened a new log
I20260709 12:17:01.975653 31771 ts_tablet_manager.cc:1403] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: Time spent bootstrapping tablet: real 0.236s	user 0.017s	sys 0.003s
I20260709 12:17:01.979503 31771 raft_consensus.cc:359] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.980376 31771 raft_consensus.cc:385] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:01.980638 31771 raft_consensus.cc:740] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Initialized, Role: FOLLOWER
I20260709 12:17:01.982803 31771 consensus_queue.cc:260] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:01.984942 31771 ts_tablet_manager.cc:1434] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:17:01.985076 32224 raft_consensus.cc:697] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:02.001587 31906 consensus_queue.cc:1048] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:01.986514 32224 consensus_queue.cc:237] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:02.014765 31399 catalog_manager.cc:5697] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:02.043447 31774 raft_consensus.cc:359] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:02.044358 31774 raft_consensus.cc:385] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:02.044698 31774 raft_consensus.cc:740] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Initialized, Role: FOLLOWER
I20260709 12:17:02.055302 31774 consensus_queue.cc:260] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:02.058512 31774 ts_tablet_manager.cc:1434] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.002s
I20260709 12:17:02.080518 32224 raft_consensus.cc:493] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:02.094210 32135 consensus_queue.cc:1048] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.098130 32224 raft_consensus.cc:515] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:02.101924 32224 leader_election.cc:290] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:02.124727 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:02.125527 31574 raft_consensus.cc:2468] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:02.127832 31691 leader_election.cc:304] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:02.129612 32120 raft_consensus.cc:2804] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:02.129989 32120 raft_consensus.cc:493] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:02.130300 32120 raft_consensus.cc:3060] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:02.134958 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:02.135941 31650 raft_consensus.cc:2468] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 0.
I20260709 12:17:02.113186 31397 catalog_manager.cc:5697] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 1 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:02.146186 32120 raft_consensus.cc:515] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:02.149607 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:02.150264 31574 raft_consensus.cc:3060] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:02.150288 31400 catalog_manager.cc:5697] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.157481 32120 leader_election.cc:290] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:02.157701 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d3dff40fad349cd90ec48c6ccc15f20" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:02.158651 31650 raft_consensus.cc:3060] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:02.157863 31400 catalog_manager.cc:5697] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 0 to 1, leader changed from <none> to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 1 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.166978 31574 raft_consensus.cc:2468] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:02.168768 31691 leader_election.cc:304] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:02.170073 32234 raft_consensus.cc:2804] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:02.170583 32234 raft_consensus.cc:697] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:02.171540 32234 consensus_queue.cc:237] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:02.174849 31650 raft_consensus.cc:2468] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:02.177577 31401 catalog_manager.cc:5697] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:02.179214 31401 catalog_manager.cc:5697] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.195976 31574 raft_consensus.cc:1275] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 4 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:02.197680 32234 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:02.202904 31980 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:02.203469 31980 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:02.205588 31980 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:02.207083 31500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:02.212726 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:02.214488 31616 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:02.215250 31980 raft_consensus.cc:2749] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:02.222818 31646 raft_consensus.cc:1275] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 4 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:02.224259 32224 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.228336 31401 catalog_manager.cc:5697] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 1 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:02.258546 31980 consensus_queue.cc:1048] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.271941 31906 consensus_queue.cc:1048] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.274542 32077 consensus_queue.cc:1048] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.306143 32135 consensus_queue.cc:1048] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.369242 31646 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:02.566239 32120 consensus_queue.cc:1048] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.569111 32224 consensus_queue.cc:1048] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:17:02.594288 32239 consensus_queue.cc:1048] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:02.633242 32224 consensus_queue.cc:1048] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:04.014130 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
W20260709 12:17:04.022119 31542 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 495 similar messages]
W20260709 12:17:04.043206 31542 consensus_peers.cc:597] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.065887 31616 consensus_peers.cc:597] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.071467 31542 consensus_peers.cc:597] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.076454 31542 consensus_peers.cc:597] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.082598 31690 consensus_peers.cc:597] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.097188 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:04.098089 22107 tablet_replica.cc:333] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.098785 22107 raft_consensus.cc:2243] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.099291 22107 raft_consensus.cc:2272] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.101433 22107 tablet_replica.cc:333] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.102178 22107 raft_consensus.cc:2243] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.102655 22107 raft_consensus.cc:2272] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.104516 31542 consensus_peers.cc:597] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.104694 22107 tablet_replica.cc:333] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.105485 22107 raft_consensus.cc:2243] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.106246 22107 raft_consensus.cc:2272] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.108016 22107 tablet_replica.cc:333] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.108729 22107 raft_consensus.cc:2243] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.109632 22107 raft_consensus.cc:2272] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.112025 22107 tablet_replica.cc:333] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.112675 22107 raft_consensus.cc:2243] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.113140 22107 raft_consensus.cc:2272] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.115276 22107 tablet_replica.cc:333] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.115937 22107 raft_consensus.cc:2243] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.116427 22107 raft_consensus.cc:2272] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.118696 22107 tablet_replica.cc:333] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.119423 22107 raft_consensus.cc:2243] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.119791 22107 raft_consensus.cc:2272] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.121577 22107 tablet_replica.cc:333] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.122215 22107 raft_consensus.cc:2243] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.122673 22107 raft_consensus.cc:2272] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.124707 22107 tablet_replica.cc:333] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.125401 22107 raft_consensus.cc:2243] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.125927 22107 raft_consensus.cc:2272] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.128325 22107 tablet_replica.cc:333] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.129002 22107 raft_consensus.cc:2243] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.129882 22107 raft_consensus.cc:2272] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.131778 22107 tablet_replica.cc:333] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.132355 22107 raft_consensus.cc:2243] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.132771 22107 raft_consensus.cc:2272] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.134598 22107 tablet_replica.cc:333] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.135195 22107 raft_consensus.cc:2243] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.135699 22107 raft_consensus.cc:2272] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.138152 22107 tablet_replica.cc:333] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.138845 22107 raft_consensus.cc:2243] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.139591 22107 raft_consensus.cc:2272] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.141626 22107 tablet_replica.cc:333] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.142251 22107 raft_consensus.cc:2243] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.142663 22107 raft_consensus.cc:2272] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.144580 22107 tablet_replica.cc:333] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.145172 22107 raft_consensus.cc:2243] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.145665 22107 raft_consensus.cc:2272] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.147446 22107 tablet_replica.cc:333] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.148125 22107 raft_consensus.cc:2243] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.148546 22107 raft_consensus.cc:2272] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.150560 22107 tablet_replica.cc:333] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.151198 22107 raft_consensus.cc:2243] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.151820 22107 raft_consensus.cc:2272] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.154620 22107 tablet_replica.cc:333] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.155401 22107 raft_consensus.cc:2243] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.156505 22107 raft_consensus.cc:2272] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.159353 22107 tablet_replica.cc:333] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.160153 22107 raft_consensus.cc:2243] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.161214 22107 raft_consensus.cc:2272] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.162431 31690 consensus_peers.cc:597] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.164024 22107 tablet_replica.cc:333] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.164776 22107 raft_consensus.cc:2243] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.165228 22107 raft_consensus.cc:2272] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.167348 22107 tablet_replica.cc:333] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.168018 22107 raft_consensus.cc:2243] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.168512 22107 raft_consensus.cc:2272] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.170789 22107 tablet_replica.cc:333] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.171365 22107 raft_consensus.cc:2243] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.171893 22107 raft_consensus.cc:2272] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.173812 22107 tablet_replica.cc:333] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.174332 22107 raft_consensus.cc:2243] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.174739 22107 raft_consensus.cc:2272] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.176568 22107 tablet_replica.cc:333] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.177066 22107 raft_consensus.cc:2243] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.177539 22107 raft_consensus.cc:2272] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.179790 22107 tablet_replica.cc:333] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.180611 22107 raft_consensus.cc:2243] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.181754 22107 raft_consensus.cc:2272] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.184501 22107 tablet_replica.cc:333] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.185348 22107 raft_consensus.cc:2243] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.185902 22107 raft_consensus.cc:2272] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.188597 22107 tablet_replica.cc:333] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.189332 22107 raft_consensus.cc:2243] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.189896 22107 raft_consensus.cc:2272] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.192112 22107 tablet_replica.cc:333] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.192809 22107 raft_consensus.cc:2243] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.193801 22107 raft_consensus.cc:2272] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.196504 22107 tablet_replica.cc:333] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.197259 22107 raft_consensus.cc:2243] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.197779 22107 raft_consensus.cc:2272] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.200137 22107 tablet_replica.cc:333] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.200901 22107 raft_consensus.cc:2243] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.201411 22107 raft_consensus.cc:2272] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.203986 22107 tablet_replica.cc:333] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.204699 22107 raft_consensus.cc:2243] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.205238 22107 raft_consensus.cc:2272] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.207880 22107 tablet_replica.cc:333] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.208638 22107 raft_consensus.cc:2243] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.209753 22107 raft_consensus.cc:2272] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.212639 22107 tablet_replica.cc:333] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
W20260709 12:17:04.212940 31616 consensus_peers.cc:597] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.213392 22107 raft_consensus.cc:2243] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.213997 22107 raft_consensus.cc:2272] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.216521 22107 tablet_replica.cc:333] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.217305 22107 raft_consensus.cc:2243] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:04.217486 31542 consensus_peers.cc:597] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.217852 22107 raft_consensus.cc:2272] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.218757 31690 consensus_peers.cc:597] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.220770 22107 tablet_replica.cc:333] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
W20260709 12:17:04.221407 31690 consensus_peers.cc:597] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.221604 31542 consensus_peers.cc:597] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.221643 22107 raft_consensus.cc:2243] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.222419 22107 raft_consensus.cc:2272] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.224431 22107 tablet_replica.cc:333] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.224939 22107 raft_consensus.cc:2243] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.225759 22107 raft_consensus.cc:2272] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.227553 22107 tablet_replica.cc:333] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.228042 22107 raft_consensus.cc:2243] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.228730 22107 raft_consensus.cc:2272] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.231124 22107 tablet_replica.cc:333] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.231776 22107 raft_consensus.cc:2243] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.232285 22107 raft_consensus.cc:2272] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.234884 31690 consensus_peers.cc:597] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.235015 22107 tablet_replica.cc:333] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.235711 22107 raft_consensus.cc:2243] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.236464 22107 raft_consensus.cc:2272] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.238639 22107 tablet_replica.cc:333] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.239177 22107 raft_consensus.cc:2243] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.239637 22107 raft_consensus.cc:2272] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.242105 22107 tablet_replica.cc:333] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.242719 22107 raft_consensus.cc:2243] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:04.243639 22107 raft_consensus.cc:2272] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.245622 22107 tablet_replica.cc:333] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.246124 22107 raft_consensus.cc:2243] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.246531 22107 raft_consensus.cc:2272] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.248365 22107 tablet_replica.cc:333] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.248843 22107 raft_consensus.cc:2243] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.249296 22107 raft_consensus.cc:2272] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.251384 22107 tablet_replica.cc:333] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.251891 22107 raft_consensus.cc:2243] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.252286 22107 raft_consensus.cc:2272] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:04.254453 22107 tablet_replica.cc:333] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: stopping tablet replica
I20260709 12:17:04.255070 22107 raft_consensus.cc:2243] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:04.255580 22107 raft_consensus.cc:2272] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:04.263590 31542 consensus_peers.cc:597] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.268360 31690 consensus_peers.cc:597] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.286346 31690 consensus_peers.cc:597] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.301198 31542 consensus_peers.cc:597] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.309414 31690 consensus_peers.cc:597] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.330507 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
W20260709 12:17:04.334769 31690 consensus_peers.cc:597] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.343806 31690 consensus_peers.cc:597] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.348542 31542 consensus_peers.cc:597] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.370785 31542 consensus_peers.cc:597] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.375563 31690 consensus_peers.cc:597] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.375571 31542 consensus_peers.cc:597] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.376238 31690 consensus_peers.cc:597] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.392737 31542 consensus_peers.cc:597] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.398028 31616 consensus_peers.cc:597] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.409622 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:04.412935 31542 consensus_peers.cc:597] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.417972 32312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:04.419622 32314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:04.420280 32311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:04.423342 31690 consensus_peers.cc:597] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.428243 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:04.432328 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:04.432591 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:04.432796 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599424432778 us; error 0 us; skew 500 ppm
I20260709 12:17:04.433374 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:04.437726 22107 webserver.cc:533] Webserver started at http://127.21.150.193:44115/ using document root <none> and password file <none>
I20260709 12:17:04.438371 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:04.438618 22107 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:17:04.443140 22107 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 12:17:04.452517 32320 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:04.453573 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:04.455063 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
format_stamp: "Formatted at 2026-07-09 12:16:50 on dist-test-slave-zctp"
I20260709 12:17:04.455626 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:17:04.458737 31542 consensus_peers.cc:597] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.530026 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:04.531701 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:04.533270 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:04.538846 32327 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 12:17:04.595355 31616 consensus_peers.cc:597] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:04.611361 31616 consensus_peers.cc:597] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:04.848657 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:17:04.849123 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.311s	user 0.008s	sys 0.000s
I20260709 12:17:04.849690 22107 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:17:04.859935 32327 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:04.877218 32327 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:04.878266 32327 tablet_bootstrap.cc:492] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:04.878896 32327 ts_tablet_manager.cc:1403] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:17:04.881891 32327 raft_consensus.cc:359] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.882611 32327 raft_consensus.cc:740] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:04.883302 32327 consensus_queue.cc:260] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.891790 32327 ts_tablet_manager.cc:1434] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.004s
I20260709 12:17:04.892643 32327 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:04.907548 32327 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:04.908466 32327 tablet_bootstrap.cc:492] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:04.909125 32327 ts_tablet_manager.cc:1403] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 12:17:04.915680 32327 raft_consensus.cc:359] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.916606 32327 raft_consensus.cc:740] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:04.917367 32327 consensus_queue.cc:260] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.919351 32327 ts_tablet_manager.cc:1434] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:04.920148 32327 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:04.937135 32327 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:04.937961 32327 tablet_bootstrap.cc:492] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:04.938411 32327 ts_tablet_manager.cc:1403] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 12:17:04.940104 32327 raft_consensus.cc:359] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.940547 32327 raft_consensus.cc:740] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:04.941321 32327 consensus_queue.cc:260] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.943490 32327 ts_tablet_manager.cc:1434] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:04.944286 32327 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:04.977924 32327 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:04.978852 32327 tablet_bootstrap.cc:492] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:04.979462 32327 ts_tablet_manager.cc:1403] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.025s
I20260709 12:17:04.981333 32327 raft_consensus.cc:359] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.982306 32327 raft_consensus.cc:740] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:04.983152 32327 consensus_queue.cc:260] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:04.985249 32327 ts_tablet_manager.cc:1434] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:04.986053 32327 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.066088 32327 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.067065 32327 tablet_bootstrap.cc:492] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.067708 32327 ts_tablet_manager.cc:1403] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.082s	user 0.031s	sys 0.044s
I20260709 12:17:05.070293 32327 raft_consensus.cc:359] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.070940 32327 raft_consensus.cc:740] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.071724 32327 consensus_queue.cc:260] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.073841 32327 ts_tablet_manager.cc:1434] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:05.074581 32327 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.093843 32327 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.094815 32327 tablet_bootstrap.cc:492] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.095432 32327 ts_tablet_manager.cc:1403] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 12:17:05.097911 32327 raft_consensus.cc:359] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.098558 32327 raft_consensus.cc:740] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.099233 32327 consensus_queue.cc:260] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.101074 32327 ts_tablet_manager.cc:1434] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:05.101850 32327 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.117148 32327 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.118081 32327 tablet_bootstrap.cc:492] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.118691 32327 ts_tablet_manager.cc:1403] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:17:05.119757 22107 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:17:05.120117 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.270s	user 0.252s	sys 0.000s
I20260709 12:17:05.120870 32327 raft_consensus.cc:359] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.121721 32327 raft_consensus.cc:740] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.122395 32327 consensus_queue.cc:260] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.124394 32327 ts_tablet_manager.cc:1434] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:05.125192 32327 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.146456 32327 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.147192 32327 tablet_bootstrap.cc:492] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.147825 32327 ts_tablet_manager.cc:1403] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.003s
I20260709 12:17:05.149901 32327 raft_consensus.cc:359] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.150426 32327 raft_consensus.cc:740] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.151026 32327 consensus_queue.cc:260] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.152514 32327 ts_tablet_manager.cc:1434] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:05.153594 32327 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.168589 32327 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.169375 32327 tablet_bootstrap.cc:492] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.169991 32327 ts_tablet_manager.cc:1403] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:17:05.171886 32327 raft_consensus.cc:359] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.172513 32327 raft_consensus.cc:740] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.173158 32327 consensus_queue.cc:260] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.174783 32327 ts_tablet_manager.cc:1434] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:05.175536 32327 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.187366 32327 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.188153 32327 tablet_bootstrap.cc:492] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.188705 32327 ts_tablet_manager.cc:1403] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:17:05.190640 32327 raft_consensus.cc:359] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.191175 32327 raft_consensus.cc:740] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.191857 32327 consensus_queue.cc:260] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.192901 32077 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.193463 32077 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.193763 32327 ts_tablet_manager.cc:1434] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 12:17:05.194577 32327 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.195334 32077 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.197103 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
W20260709 12:17:05.219754 31616 leader_election.cc:336] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.220387 31616 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:05.222810 32299 raft_consensus.cc:2749] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.224498 32307 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.225288 32307 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.237076 32327 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.230464 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:05.228129 32307 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:05.246184 32327 tablet_bootstrap.cc:492] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.250298 32327 ts_tablet_manager.cc:1403] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.020s
W20260709 12:17:05.253051 31542 leader_election.cc:336] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.253116 32327 raft_consensus.cc:359] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.256263 32327 raft_consensus.cc:740] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.257148 32327 consensus_queue.cc:260] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.258562 32077 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.259037 32077 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.259627 32327 ts_tablet_manager.cc:1434] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:17:05.260433 32327 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.260857 32077 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.263564 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.264405 31574 raft_consensus.cc:2468] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.265861 31617 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:05.266971 32077 raft_consensus.cc:2804] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.267480 32077 raft_consensus.cc:493] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.268087 32077 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.277753 31545 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:05.278908 32307 raft_consensus.cc:2749] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.278872 32077 raft_consensus.cc:515] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.282382 32302 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.283114 32302 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.286710 32077 leader_election.cc:290] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.287813 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.288549 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd9c92a3ff84f17a24aeceaffae4ef9" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:05.289134 31579 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.291040 32302 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
W20260709 12:17:05.294893 31616 leader_election.cc:336] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
W20260709 12:17:05.298645 31616 leader_election.cc:336] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.299026 32327 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:17:05.299690 31690 leader_election.cc:336] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.299955 32327 tablet_bootstrap.cc:492] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.300320 31690 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:05.300740 32327 ts_tablet_manager.cc:1403] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.011s
I20260709 12:17:05.301278 32302 raft_consensus.cc:2749] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:05.303212 32327 raft_consensus.cc:359] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.303716 32327 raft_consensus.cc:740] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.304272 32327 consensus_queue.cc:260] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.305104 32077 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.305598 32077 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.307168 32327 ts_tablet_manager.cc:1434] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:17:05.307304 32077 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.308014 32327 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.308190 32302 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.308516 31579 raft_consensus.cc:2468] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:05.308660 32302 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.310619 31617 leader_election.cc:304] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:05.311894 32307 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.312450 32307 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.313558 32299 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.314100 32299 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.314440 32307 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.318611 32299 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.319140 32299 raft_consensus.cc:2804] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.319696 32299 raft_consensus.cc:697] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:05.320290 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.320484 32299 consensus_queue.cc:237] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.323827 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.327845 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.328859 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.331452 32302 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
W20260709 12:17:05.331924 31542 leader_election.cc:336] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.333837 31542 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:05.335417 31906 raft_consensus.cc:2749] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:05.331939 31690 leader_election.cc:336] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.333153 32327 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.336791 32327 tablet_bootstrap.cc:492] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.337610 32327 ts_tablet_manager.cc:1403] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.001s
I20260709 12:17:05.337877 31691 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:05.339244 32302 raft_consensus.cc:2749] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.344059 32327 raft_consensus.cc:359] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.346488 32327 raft_consensus.cc:740] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.347966 32327 consensus_queue.cc:260] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.350874 32327 ts_tablet_manager.cc:1434] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:17:05.351819 32327 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.365005 31906 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.367493 31906 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.369815 31906 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.370429 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.371091 31721 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:05.374972 31542 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.375855 31906 raft_consensus.cc:2804] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.376190 31906 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.376500 31906 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:05.381593 31542 leader_election.cc:336] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
W20260709 12:17:05.383092 31616 leader_election.cc:336] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.383597 31616 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:05.384385 32316 raft_consensus.cc:2749] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:05.384501 31616 leader_election.cc:336] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:05.384943 31616 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:05.385720 32316 raft_consensus.cc:2749] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.389602 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:37911
I20260709 12:17:05.389185 32402 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:37911 every 8 connection(s)
I20260709 12:17:05.393131 31906 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.395267 31906 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.395480 32327 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.396296 32327 tablet_bootstrap.cc:492] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.396950 32327 ts_tablet_manager.cc:1403] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.045s	user 0.014s	sys 0.003s
I20260709 12:17:05.398743 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:05.399345 31721 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.399523 32327 raft_consensus.cc:359] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.400241 32327 raft_consensus.cc:740] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.400907 32327 consensus_queue.cc:260] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.403035 32327 ts_tablet_manager.cc:1434] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:17:05.402959 32302 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.403617 32302 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.404001 32327 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:05.410418 32302 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.411509 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.410959 31401 catalog_manager.cc:5697] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:05.412174 31579 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:05.413353 31691 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.414160 32224 raft_consensus.cc:2804] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.414499 32224 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.414834 32224 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.419945 31721 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:17:05.421332 31542 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.421761 32307 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.422320 31906 raft_consensus.cc:2804] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.422261 32307 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.423609 31906 raft_consensus.cc:697] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:05.424507 32307 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:05.424422 31906 consensus_queue.cc:237] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.427814 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:05.432597 32224 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.443490 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:05.444116 31579 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.453052 32224 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.455361 31579 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:05.466107 31691 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.463157 32369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.460037 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:17:05.468935 31542 leader_election.cc:343] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.469578 31542 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:05.469750 32224 raft_consensus.cc:2804] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:05.470564 32307 raft_consensus.cc:2749] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:05.471560 32224 raft_consensus.cc:697] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
W20260709 12:17:05.472685 31542 leader_election.cc:343] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.478536 32224 consensus_queue.cc:237] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.494469 32307 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.495348 32307 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.501394 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
W20260709 12:17:05.503573 31542 leader_election.cc:343] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.505661 31399 catalog_manager.cc:5697] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 2 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:05.512341 32302 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.539280 32302 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.515619 32400 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.541651 32400 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.544389 32400 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.516076 32299 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.545126 32299 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.550534 32368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.551638 32368 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.552971 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.555748 31616 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
W20260709 12:17:05.556800 31616 leader_election.cc:343] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.557507 32316 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.558166 32316 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.560752 32316 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.515928 31397 catalog_manager.cc:5697] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:05.522292 32239 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.714208 32365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.568267 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.717886 32367 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.712710 32414 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.719472 32414 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.719884 32422 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.719990 32239 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.601207 32224 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.729588 32422 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.729766 32224 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.731738 32422 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.731750 32224 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.561333 32316 raft_consensus.cc:2804] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.758730 32077 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.759373 32077 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.757247 31616 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:05.762529 32414 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.783324 32426 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.784679 32426 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.797988 32077 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.798596 32077 raft_consensus.cc:2804] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.798930 32077 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.799340 32077 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.801028 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.801921 31727 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.628078 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.754869 32299 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.809770 32327 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:05.813625 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:05.814482 31579 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.776475 32316 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.785566 32422 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.806365 32307 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:05.832738 32327 tablet_bootstrap.cc:492] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:05.833526 32327 ts_tablet_manager.cc:1403] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.430s	user 0.037s	sys 0.045s
I20260709 12:17:05.838516 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:05.839418 31652 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:05.840425 32425 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.840993 32425 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.819669 32316 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.843567 32425 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.813663 31721 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:05.858325 32239 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.827782 32422 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.851562 32368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:05.854689 32302 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.863051 32426 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.850443 32427 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.863915 32427 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.866034 32364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.866842 32364 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:05.867326 32365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.867926 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.868721 32363 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:05.869683 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.868001 32365 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
W20260709 12:17:05.873231 31542 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:05.887421 31690 leader_election.cc:343] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.879868 32327 raft_consensus.cc:359] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.889710 32327 raft_consensus.cc:740] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:05.890697 32327 consensus_queue.cc:260] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:05.893534 32327 ts_tablet_manager.cc:1434] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 12:17:05.897094 32327 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:17:05.895828 31690 leader_election.cc:343] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:05.894069 32077 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.930511 31690 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.935596 32077 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.943081 32422 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.943312 32431 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.943902 32431 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.939177 31690 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:05.947814 31727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.948617 31727 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:17:05.950637 31617 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:05.938083 32364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.956128 32077 raft_consensus.cc:2804] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.944635 32361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:05.956526 32077 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.959998 32077 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:05.946425 32426 raft_consensus.cc:2804] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.945302 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:05.961648 31616 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:05.945533 32316 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:05.981352 32077 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:05.972677 32403 heartbeater.cc:344] Connected to a master server at 127.21.150.254:39763
I20260709 12:17:05.963047 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:05.965116 32426 raft_consensus.cc:493] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.965238 32427 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:05.966039 32439 raft_consensus.cc:2804] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.966138 32420 raft_consensus.cc:2804] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:05.967092 32316 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:05.967861 32361 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:05.996945 32426 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.001820 32077 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:05.997310 32403 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:06.005932 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:06.004701 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:06.006578 31579 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.007182 31574 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:06.003137 32420 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:05.999364 32439 raft_consensus.cc:493] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.000170 32364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:06.011835 32420 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.017798 32403 heartbeater.cc:507] Master 127.21.150.254:39763 requested a full tablet report, sending...
I20260709 12:17:06.022181 32431 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:06.023056 32368 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:06.024433 32365 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.025019 32365 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.026971 32439 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.027840 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.029584 32361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:06.029667 32369 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:06.031692 31542 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.032737 32425 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:06.032956 32362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.033125 32425 raft_consensus.cc:493] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.033623 32425 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.037241 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:06.037904 31722 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
W20260709 12:17:06.039814 31542 leader_election.cc:343] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.041520 31574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:06.033641 32362 raft_consensus.cc:3060] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:06.043141 31616 leader_election.cc:343] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.043793 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:06.044139 31690 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.037129 32420 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.024912 31545 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
W20260709 12:17:06.049400 31542 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.059907 31542 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.062809 32436 raft_consensus.cc:2804] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.063220 32436 raft_consensus.cc:493] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.063562 32436 raft_consensus.cc:3060] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.066361 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:06.075794 31691 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.076900 32437 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.077256 32437 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.077607 32437 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.048946 31578 raft_consensus.cc:1275] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.050369 31616 leader_election.cc:343] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.048607 31574 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
W20260709 12:17:06.079257 31616 consensus_peers.cc:597] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.079378 31906 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.081604 31906 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.081948 31906 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.084333 32424 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.084771 32363 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
W20260709 12:17:06.052860 31690 leader_election.cc:343] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:06.085975 31690 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.086735 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.087510 31650 raft_consensus.cc:2393] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:06.087657 31690 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.088593 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.088959 32422 raft_consensus.cc:2804] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.089385 32422 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.089754 32422 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.089867 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.090672 31652 raft_consensus.cc:2393] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:06.090567 31646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.091241 31646 raft_consensus.cc:2468] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:06.084790 32424 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.093950 32424 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.094532 31617 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.095270 32316 consensus_queue.cc:1048] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.096580 32316 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.096905 32316 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.099504 31721 raft_consensus.cc:2393] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 2: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:06.101599 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.102216 31652 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:06.097229 32316 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.104094 32420 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.104367 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d" is_pre_election: true
I20260709 12:17:06.105064 31649 raft_consensus.cc:2468] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 1.
I20260709 12:17:06.108479 31545 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.109284 32417 raft_consensus.cc:2804] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.109661 32417 raft_consensus.cc:493] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.110006 32417 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.111913 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.113910 32367 raft_consensus.cc:3060] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.115320 32439 raft_consensus.cc:515] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.116492 31650 raft_consensus.cc:2393] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:06.116331 32425 raft_consensus.cc:515] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.117497 32439 leader_election.cc:290] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.057005 31579 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
W20260709 12:17:06.114113 31690 consensus_peers.cc:597] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.121292 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.121915 31617 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.124138 31579 raft_consensus.cc:1275] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.125814 31542 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.150.196:40451, user_credentials={real_user=slave}} blocked reactor thread for 65975.4us
I20260709 12:17:06.132213 31721 raft_consensus.cc:1275] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.132481 32316 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.132794 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.133677 31649 raft_consensus.cc:2393] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:06.130227 31906 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.126370 32426 raft_consensus.cc:515] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.135841 31718 raft_consensus.cc:2393] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 2: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
W20260709 12:17:06.136974 31542 consensus_peers.cc:597] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.138476 31906 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:06.140787 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.140980 31721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28ff8e905a7b4eb5b2cea1df313efa14" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:06.141289 32316 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:06.141671 31649 raft_consensus.cc:2393] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 2: Already voted for candidate 0bd0f6deb3b147509a008c574d9d879d in this term.
I20260709 12:17:06.141515 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.144646 32426 leader_election.cc:290] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.144778 32436 raft_consensus.cc:515] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.141696 31721 raft_consensus.cc:2393] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 2: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:06.147135 31722 raft_consensus.cc:2393] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 2: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:06.137214 32442 raft_consensus.cc:2804] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.143875 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:06.127466 32418 consensus_queue.cc:1048] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
W20260709 12:17:06.146667 31616 leader_election.cc:343] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.150278 31579 raft_consensus.cc:2393] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bd0f6deb3b147509a008c574d9d879d in current term 2: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:17:06.137095 32365 raft_consensus.cc:2468] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:06.145974 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.153134 32414 consensus_queue.cc:1048] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.153882 32425 leader_election.cc:290] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.155848 31617 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:06.153191 32436 leader_election.cc:290] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.157872 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.158164 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.158511 32417 raft_consensus.cc:515] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.160987 32367 raft_consensus.cc:2468] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:06.162398 32442 raft_consensus.cc:697] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:06.163491 32442 consensus_queue.cc:237] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.164918 31616 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:06.166221 32362 raft_consensus.cc:2468] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:06.167410 32422 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.169765 32439 raft_consensus.cc:2804] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.169929 31616 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:06.170533 32439 raft_consensus.cc:697] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:06.170775 32441 raft_consensus.cc:2804] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.171319 32441 raft_consensus.cc:697] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:06.172179 32441 consensus_queue.cc:237] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.178719 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:06.180168 31722 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:06.163950 32443 raft_consensus.cc:2749] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.190269 32362 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:17:06.186064 31542 leader_election.cc:343] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.191985 31542 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.197860 32414 raft_consensus.cc:2749] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.195004 32439 consensus_queue.cc:237] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.191974 31690 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.197000 31722 raft_consensus.cc:2468] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 3.
I20260709 12:17:06.202770 32420 raft_consensus.cc:2804] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.203423 32420 raft_consensus.cc:697] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:06.204324 32420 consensus_queue.cc:237] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.185626 31693 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.210822 32421 raft_consensus.cc:2804] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:06.211203 32421 raft_consensus.cc:493] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:06.212339 32362 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.211556 32421 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.212515 31542 leader_election.cc:304] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.221479 31906 raft_consensus.cc:2804] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:06.223619 31906 raft_consensus.cc:697] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:06.233008 32417 leader_election.cc:290] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:06.236773 31906 consensus_queue.cc:237] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.178056 32422 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.245393 31616 leader_election.cc:304] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:06.246423 32443 raft_consensus.cc:2749] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.249464 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.250226 31650 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.158535 32437 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.226898 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfd13293d3149f6a1ca73fd10852083" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.262830 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c6ef1dbe6184d41b2a5edfb45c8315e" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.266677 31652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.267467 31652 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.262015 31650 raft_consensus.cc:2468] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:17:06.278851 31545 leader_election.cc:304] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:06.283783 31652 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:06.286206 31693 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:06.292528 31693 leader_election.cc:304] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.295207 32239 raft_consensus.cc:2749] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:06.297291 31542 leader_election.cc:343] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.290637 32417 raft_consensus.cc:2804] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.295890 32426 raft_consensus.cc:2804] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.351647 32327 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.289834 32424 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.346994 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68cd2a3bb89a49a6baa7af253a090766" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.349038 32417 raft_consensus.cc:697] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:06.359519 32426 raft_consensus.cc:697] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:06.364975 32426 consensus_queue.cc:237] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.367631 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc169af544e14e668b85ec4da6ca648d" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.372752 31693 leader_election.cc:304] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.380219 32327 tablet_bootstrap.cc:492] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.381278 32327 ts_tablet_manager.cc:1403] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.484s	user 0.018s	sys 0.005s
I20260709 12:17:06.384169 32437 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:06.345808 32421 raft_consensus.cc:515] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.384981 32363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:06.382128 32422 raft_consensus.cc:2749] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.386616 32417 consensus_queue.cc:237] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.393906 32362 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:06.401325 31399 ts_manager.cc:194] Re-registered known tserver with Master: 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.404467 32421 leader_election.cc:290] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.401930 32367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:17:06.407305 31690 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.374850 32424 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:06.407788 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
W20260709 12:17:06.409628 31690 leader_election.cc:343] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.416170 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d00a005dd8945d9acac25d9822223a6" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:06.416801 31649 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.418285 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "bf0f149240cc4f27b97810c90eba7ff9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:06.419065 31578 raft_consensus.cc:2393] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bf0f149240cc4f27b97810c90eba7ff9 in current term 2: Already voted for candidate 843512b2aa644b55a21b7e702943e02f in this term.
I20260709 12:17:06.421391 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:06.422195 31722 raft_consensus.cc:2393] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 843512b2aa644b55a21b7e702943e02f in current term 2: Already voted for candidate bf0f149240cc4f27b97810c90eba7ff9 in this term.
I20260709 12:17:06.427591 32327 raft_consensus.cc:359] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.428550 32327 raft_consensus.cc:740] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.429267 32327 consensus_queue.cc:260] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.431334 32327 ts_tablet_manager.cc:1434] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.000s
I20260709 12:17:06.432140 32327 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.432188 31691 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:06.433182 32224 raft_consensus.cc:2749] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:06.434156 31542 leader_election.cc:343] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.435942 31397 catalog_manager.cc:5697] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.437556 31397 catalog_manager.cc:5697] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:06.438998 31397 catalog_manager.cc:5697] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:06.440889 31542 leader_election.cc:343] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:06.450865 31542 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 843512b2aa644b55a21b7e702943e02f; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9
I20260709 12:17:06.453132 32429 raft_consensus.cc:2749] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:06.458765 31649 raft_consensus.cc:2468] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf0f149240cc4f27b97810c90eba7ff9 in term 2.
I20260709 12:17:06.460916 31693 leader_election.cc:304] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:06.462448 32426 raft_consensus.cc:2804] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:06.466964 32426 raft_consensus.cc:697] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Leader. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:06.470288 32426 consensus_queue.cc:237] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.473685 32327 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.474634 32327 tablet_bootstrap.cc:492] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.475275 32327 ts_tablet_manager.cc:1403] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.003s
I20260709 12:17:06.477239 32327 raft_consensus.cc:359] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.477874 32327 raft_consensus.cc:740] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.478641 32327 consensus_queue.cc:260] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.480829 32327 ts_tablet_manager.cc:1434] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:06.491781 32327 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.507692 31401 catalog_manager.cc:5697] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 2 to 3, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 3 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:06.510097 31401 catalog_manager.cc:5697] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 2 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:06.517583 32327 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.518606 32327 tablet_bootstrap.cc:492] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.519321 32327 ts_tablet_manager.cc:1403] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.003s
I20260709 12:17:06.521958 32327 raft_consensus.cc:359] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.522637 32327 raft_consensus.cc:740] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.523458 32327 consensus_queue.cc:260] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.525439 32327 ts_tablet_manager.cc:1434] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:06.526315 32327 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.527904 32362 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:06.529595 31400 catalog_manager.cc:5697] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:06.532366 31400 catalog_manager.cc:5697] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:06.541579 32327 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.542817 32327 tablet_bootstrap.cc:492] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
W20260709 12:17:06.543777 31542 consensus_peers.cc:597] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.545804 32327 ts_tablet_manager.cc:1403] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.002s
I20260709 12:17:06.548414 32327 raft_consensus.cc:359] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.549113 32327 raft_consensus.cc:740] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.549955 32327 consensus_queue.cc:260] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.552225 32327 ts_tablet_manager.cc:1434] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:06.553031 32327 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.568311 32327 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.569267 32327 tablet_bootstrap.cc:492] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.569895 32327 ts_tablet_manager.cc:1403] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:17:06.572360 32327 raft_consensus.cc:359] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.572976 32327 raft_consensus.cc:740] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.573678 32327 consensus_queue.cc:260] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:06.575304 32327 ts_tablet_manager.cc:1434] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:17:06.575934 32327 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:17:06.587391 31616 consensus_peers.cc:597] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.589335 31400 catalog_manager.cc:5697] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 reported cstate change: term changed from 1 to 2, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196). New cstate: current_term: 2 leader_uuid: "bf0f149240cc4f27b97810c90eba7ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } }
W20260709 12:17:06.597854 31616 consensus_peers.cc:597] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.602124 32327 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.604167 32327 tablet_bootstrap.cc:492] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.604907 31722 raft_consensus.cc:1275] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.605712 32327 ts_tablet_manager.cc:1403] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.005s
I20260709 12:17:06.607566 32442 consensus_queue.cc:1048] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.608302 32327 raft_consensus.cc:359] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.608989 32327 raft_consensus.cc:740] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.609843 32327 consensus_queue.cc:260] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.611961 32327 ts_tablet_manager.cc:1434] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:06.612746 32327 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.622514 31399 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53000
I20260709 12:17:06.627489 32403 heartbeater.cc:499] Master 127.21.150.254:39763 was elected leader, sending a full tablet report...
I20260709 12:17:06.629596 32362 raft_consensus.cc:1275] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.632651 32327 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.633913 32327 tablet_bootstrap.cc:492] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.634984 32327 ts_tablet_manager.cc:1403] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 12:17:06.634972 32439 consensus_queue.cc:1048] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:06.638230 32327 raft_consensus.cc:359] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.638927 32327 raft_consensus.cc:740] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.639599 32327 consensus_queue.cc:260] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.641214 32327 ts_tablet_manager.cc:1434] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:06.642014 32327 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.662848 32327 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.663757 32327 tablet_bootstrap.cc:492] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.664368 32327 ts_tablet_manager.cc:1403] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:17:06.669982 32362 raft_consensus.cc:1275] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.670189 31542 consensus_peers.cc:597] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.671479 32327 raft_consensus.cc:359] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.671751 32316 consensus_queue.cc:1048] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.672240 32327 raft_consensus.cc:740] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.673028 32327 consensus_queue.cc:260] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.675117 32327 ts_tablet_manager.cc:1434] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:17:06.675901 32327 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.677469 32365 raft_consensus.cc:1275] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.677948 31646 raft_consensus.cc:1275] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.679515 32421 consensus_queue.cc:1048] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:06.681300 31690 consensus_peers.cc:597] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.682945 32420 consensus_queue.cc:1048] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.693585 31722 raft_consensus.cc:1275] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:06.698419 32417 consensus_queue.cc:1048] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:06.708626 31721 raft_consensus.cc:1275] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.710045 32327 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:17:06.710587 31542 consensus_peers.cc:597] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.711082 32327 tablet_bootstrap.cc:492] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.711781 32327 ts_tablet_manager.cc:1403] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.008s
I20260709 12:17:06.718235 32316 consensus_queue.cc:1048] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:06.721288 32327 raft_consensus.cc:359] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.722038 32327 raft_consensus.cc:740] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:06.725189 32327 consensus_queue.cc:260] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:06.728780 32327 ts_tablet_manager.cc:1434] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:17:06.729763 32327 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.750916 31578 raft_consensus.cc:1275] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.753818 31690 consensus_peers.cc:597] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.758271 32327 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:06.759207 32327 tablet_bootstrap.cc:492] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:06.759912 32327 ts_tablet_manager.cc:1403] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.003s
I20260709 12:17:06.832324 32363 raft_consensus.cc:1275] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.842211 32442 consensus_queue.cc:1048] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:17:06.848172 32433 consensus_queue.cc:1048] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:06.762708 32327 raft_consensus.cc:359] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:06.880235 32327 raft_consensus.cc:740] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
W20260709 12:17:06.889786 31690 consensus_peers.cc:597] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.902145 32327 consensus_queue.cc:260] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
W20260709 12:17:06.911666 31542 consensus_peers.cc:597] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:06.912652 31542 consensus_peers.cc:597] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:06.913375 31542 consensus_peers.cc:597] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:06.927975 32327 ts_tablet_manager.cc:1434] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.168s	user 0.007s	sys 0.004s
I20260709 12:17:06.966898 31649 raft_consensus.cc:1275] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:06.969221 32437 consensus_queue.cc:1048] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:06.978911 32327 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:06.986363 31651 raft_consensus.cc:1275] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:17:06.982582 31542 consensus_peers.cc:597] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:06.997995 32414 consensus_queue.cc:1048] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:07.013311 31690 consensus_peers.cc:597] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:07.038635 31542 consensus_peers.cc:597] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:07.055153 31690 consensus_peers.cc:597] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:07.055871 31690 consensus_peers.cc:597] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:07.057055 31542 consensus_peers.cc:597] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:07.082693 31690 consensus_peers.cc:597] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:07.093319 32327 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.099614 31649 raft_consensus.cc:1275] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer bf0f149240cc4f27b97810c90eba7ff9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:07.101961 32327 tablet_bootstrap.cc:492] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.102845 32327 ts_tablet_manager.cc:1403] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.167s	user 0.026s	sys 0.008s
I20260709 12:17:07.102994 32426 consensus_queue.cc:1048] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:07.105787 32327 raft_consensus.cc:359] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.106576 32327 raft_consensus.cc:740] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.107334 32327 consensus_queue.cc:260] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.109937 32327 ts_tablet_manager.cc:1434] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:07.110845 32327 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.135828 32327 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.136616 32327 tablet_bootstrap.cc:492] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.137202 32327 ts_tablet_manager.cc:1403] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.004s
I20260709 12:17:07.139632 32327 raft_consensus.cc:359] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:07.140268 32327 raft_consensus.cc:740] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.140937 32327 consensus_queue.cc:260] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:07.146349 32327 ts_tablet_manager.cc:1434] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:17:07.147272 32327 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
W20260709 12:17:07.163096 31690 consensus_peers.cc:597] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:07.163502 32327 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.164402 32327 tablet_bootstrap.cc:492] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.164883 32327 ts_tablet_manager.cc:1403] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:17:07.167609 32327 raft_consensus.cc:359] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.168366 32327 raft_consensus.cc:740] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.169102 32327 consensus_queue.cc:260] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.171310 32327 ts_tablet_manager.cc:1434] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:07.172228 32327 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.193568 32327 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.194442 32327 tablet_bootstrap.cc:492] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.195179 32327 ts_tablet_manager.cc:1403] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.002s
I20260709 12:17:07.197781 32327 raft_consensus.cc:359] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.198488 32327 raft_consensus.cc:740] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.199290 32327 consensus_queue.cc:260] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.201570 32327 ts_tablet_manager.cc:1434] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:07.202498 32327 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.218981 32327 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.219897 32327 tablet_bootstrap.cc:492] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.220533 32327 ts_tablet_manager.cc:1403] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 12:17:07.223158 32327 raft_consensus.cc:359] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.223815 32327 raft_consensus.cc:740] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.224488 32327 consensus_queue.cc:260] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.226632 32327 ts_tablet_manager.cc:1434] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:07.228180 32327 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.240288 32327 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.241055 32327 tablet_bootstrap.cc:492] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.241688 32327 ts_tablet_manager.cc:1403] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 12:17:07.244319 32327 raft_consensus.cc:359] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.244980 32327 raft_consensus.cc:740] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.245618 32327 consensus_queue.cc:260] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.247428 32327 ts_tablet_manager.cc:1434] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:07.248456 32327 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.281096 32365 raft_consensus.cc:3060] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.283161 32327 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.284274 32327 tablet_bootstrap.cc:492] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.284955 32327 ts_tablet_manager.cc:1403] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.037s	user 0.007s	sys 0.011s
I20260709 12:17:07.287870 32327 raft_consensus.cc:359] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.288543 32327 raft_consensus.cc:740] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.289202 32327 consensus_queue.cc:260] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.290961 32327 ts_tablet_manager.cc:1434] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:17:07.291745 32327 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.307221 32327 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.308184 32327 tablet_bootstrap.cc:492] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.308835 32327 ts_tablet_manager.cc:1403] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 12:17:07.311393 32327 raft_consensus.cc:359] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.312062 32327 raft_consensus.cc:740] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.312631 32327 consensus_queue.cc:260] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.314788 32327 ts_tablet_manager.cc:1434] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:17:07.315932 32327 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.332702 32327 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.333653 32327 tablet_bootstrap.cc:492] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.334306 32327 ts_tablet_manager.cc:1403] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 12:17:07.336403 32327 raft_consensus.cc:359] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.336899 32327 raft_consensus.cc:740] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.337636 32327 consensus_queue.cc:260] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.340477 32327 ts_tablet_manager.cc:1434] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:07.341769 32327 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.356521 32327 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.357599 32327 tablet_bootstrap.cc:492] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.358325 32327 ts_tablet_manager.cc:1403] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:17:07.361383 32327 raft_consensus.cc:359] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.362075 32327 raft_consensus.cc:740] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.362753 32327 consensus_queue.cc:260] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.365319 32327 ts_tablet_manager.cc:1434] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:07.366690 32327 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.382862 32327 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.384105 32327 tablet_bootstrap.cc:492] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.384871 32327 ts_tablet_manager.cc:1403] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.001s
I20260709 12:17:07.388551 32327 raft_consensus.cc:359] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.389068 32327 raft_consensus.cc:740] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.389770 32327 consensus_queue.cc:260] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.391685 32327 ts_tablet_manager.cc:1434] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:17:07.392860 32327 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.415719 32327 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.416481 32327 tablet_bootstrap.cc:492] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.417129 32327 ts_tablet_manager.cc:1403] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 12:17:07.420002 32327 raft_consensus.cc:359] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.420678 32327 raft_consensus.cc:740] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.422363 32327 consensus_queue.cc:260] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.424479 32327 ts_tablet_manager.cc:1434] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:07.425271 32327 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.441859 32327 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.442821 32327 tablet_bootstrap.cc:492] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.443383 32327 ts_tablet_manager.cc:1403] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 12:17:07.445394 32327 raft_consensus.cc:359] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.445952 32327 raft_consensus.cc:740] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.446523 32327 consensus_queue.cc:260] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.448614 32327 ts_tablet_manager.cc:1434] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:07.449434 32327 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.466246 32327 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.467397 32327 tablet_bootstrap.cc:492] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.468037 32327 ts_tablet_manager.cc:1403] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:17:07.470654 32327 raft_consensus.cc:359] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.471284 32327 raft_consensus.cc:740] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.471936 32327 consensus_queue.cc:260] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.474195 32327 ts_tablet_manager.cc:1434] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:07.482968 32327 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.503224 32327 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.504128 32327 tablet_bootstrap.cc:492] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.504771 32327 ts_tablet_manager.cc:1403] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.003s
I20260709 12:17:07.507570 32327 raft_consensus.cc:359] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.512259 32327 raft_consensus.cc:740] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.512980 32327 consensus_queue.cc:260] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.515053 32327 ts_tablet_manager.cc:1434] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:17:07.517001 32327 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.528913 32365 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.544853 32327 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.545904 32327 tablet_bootstrap.cc:492] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.546586 32327 ts_tablet_manager.cc:1403] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.030s	user 0.020s	sys 0.006s
I20260709 12:17:07.549265 32327 raft_consensus.cc:359] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.549937 32327 raft_consensus.cc:740] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.550685 32327 consensus_queue.cc:260] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.553613 32327 ts_tablet_manager.cc:1434] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:07.554491 32327 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.568080 32327 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.568861 32327 tablet_bootstrap.cc:492] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.569568 32327 ts_tablet_manager.cc:1403] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 12:17:07.572368 32327 raft_consensus.cc:359] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.573173 32327 raft_consensus.cc:740] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.573868 32327 consensus_queue.cc:260] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.576009 32327 ts_tablet_manager.cc:1434] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:07.577491 32327 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.592581 32327 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.593326 32327 tablet_bootstrap.cc:492] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.593883 32327 ts_tablet_manager.cc:1403] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:17:07.596041 32327 raft_consensus.cc:359] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.596716 32327 raft_consensus.cc:740] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.597324 32327 consensus_queue.cc:260] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:07.599809 32327 ts_tablet_manager.cc:1434] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:07.600616 32327 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.615844 32327 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.616762 32327 tablet_bootstrap.cc:492] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.617378 32327 ts_tablet_manager.cc:1403] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:17:07.620175 32327 raft_consensus.cc:359] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.620820 32327 raft_consensus.cc:740] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.621596 32327 consensus_queue.cc:260] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.624094 32327 ts_tablet_manager.cc:1434] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:07.625555 32327 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap starting.
I20260709 12:17:07.640406 32327 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:07.641333 32327 tablet_bootstrap.cc:492] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Bootstrap complete.
I20260709 12:17:07.642652 32327 ts_tablet_manager.cc:1403] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:17:07.644883 32327 raft_consensus.cc:359] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.645447 32327 raft_consensus.cc:740] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Initialized, Role: FOLLOWER
I20260709 12:17:07.646106 32327 consensus_queue.cc:260] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:07.649055 32327 ts_tablet_manager.cc:1434] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:17:07.689591 32365 raft_consensus.cc:3060] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.694950 32370 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:07.768916 32442 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:07.769570 32442 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.773545 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:07.773916 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f" is_pre_election: true
I20260709 12:17:07.774477 32370 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 1.
I20260709 12:17:07.774720 31578 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:07.776062 31616 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:07.776253 32442 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:07.777040 32441 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:07.777458 32441 raft_consensus.cc:493] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:07.777779 32441 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.784252 32441 raft_consensus.cc:515] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.786741 32441 leader_election.cc:290] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 election: Requested vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:07.787422 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:07.787453 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5dfd0d3d6f941e68232bcc32722b5de" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:07.788089 31578 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.788233 32370 raft_consensus.cc:3060] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:07.794112 31578 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 3.
I20260709 12:17:07.794451 32370 raft_consensus.cc:2468] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 3.
I20260709 12:17:07.795500 31617 leader_election.cc:304] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:07.796319 32441 raft_consensus.cc:2804] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:07.797079 32441 raft_consensus.cc:697] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 3 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:07.798251 32441 consensus_queue.cc:237] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:07.818615 31397 catalog_manager.cc:5697] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 3, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 3 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:07.911029 32365 raft_consensus.cc:3060] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:08.021816 32370 raft_consensus.cc:3060] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:08.182336 32414 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:08.182847 32414 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:08.184962 32414 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:08.186025 31719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9" is_pre_election: true
I20260709 12:17:08.186059 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" is_pre_election: true
I20260709 12:17:08.186837 31719 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 2.
I20260709 12:17:08.186928 32370 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 1.
I20260709 12:17:08.188130 31542 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:08.188881 32414 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:08.189188 32414 raft_consensus.cc:493] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:08.189513 32414 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:08.194886 32414 raft_consensus.cc:515] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:08.196667 32414 leader_election.cc:290] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:08.197513 31719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:08.197613 32370 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6019e17e78a14e9fa70a4f5aa6e26056" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
I20260709 12:17:08.198307 31719 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:08.199331 32370 raft_consensus.cc:3060] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:08.208081 31719 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 3.
I20260709 12:17:08.208760 32370 raft_consensus.cc:2468] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 3.
I20260709 12:17:08.209944 31542 leader_election.cc:304] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:08.210605 32414 raft_consensus.cc:2804] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:08.211863 32414 raft_consensus.cc:697] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:08.212643 32414 consensus_queue.cc:237] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:08.222064 31401 catalog_manager.cc:5697] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 1 to 3, leader changed from 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193) to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 3 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:08.240365 32370 raft_consensus.cc:1275] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:08.241688 32316 consensus_queue.cc:1048] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.251506 31579 raft_consensus.cc:1275] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:08.254350 32442 consensus_queue.cc:1048] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:08.796911 32370 raft_consensus.cc:1275] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:08.798317 32521 consensus_queue.cc:1048] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:08.822153 31719 raft_consensus.cc:1275] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:08.825484 32522 consensus_queue.cc:1048] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:15.496425 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:15.557327 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:15.558176 31578 raft_consensus.cc:606] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:15.560716 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:15.561268 31578 raft_consensus.cc:606] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:15.563788 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ee706eb478f44e0979857553c460e94"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:15.564258 31578 raft_consensus.cc:606] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:15.566752 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:15.567296 31578 raft_consensus.cc:606] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:15.569826 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:15.570327 31578 raft_consensus.cc:606] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:15.583011 31718 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:15.583536 31718 raft_consensus.cc:606] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:15.584671 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:17:15.586002 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:15.586742 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:15.641721 32543 raft_consensus.cc:993] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.642243 32543 raft_consensus.cc:1081] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.643707 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:15.644168 32363 raft_consensus.cc:493] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:15.644412 32363 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.650765 32363 raft_consensus.cc:515] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.659040 32363 leader_election.cc:290] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:15.672406 32543 raft_consensus.cc:993] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.673002 32543 raft_consensus.cc:1081] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.675148 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:15.675726 31578 raft_consensus.cc:3055] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.675750 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4ba9fa8e8004145a227359512e2443d" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:15.675011 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:15.676115 31578 raft_consensus.cc:740] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:15.676415 31649 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.676479 32363 raft_consensus.cc:493] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:15.677078 32363 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.684422 32363 raft_consensus.cc:515] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.688182 32363 leader_election.cc:290] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:15.688700 31649 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.688879 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:15.689658 31579 raft_consensus.cc:3055] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:15.689960 31579 raft_consensus.cc:740] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:15.690750 31579 consensus_queue.cc:260] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.690341 32324 leader_election.cc:304] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:15.691752 31579 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.693101 32557 raft_consensus.cc:2804] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:15.693799 31578 consensus_queue.cc:260] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.707345 31578 raft_consensus.cc:3060] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.713792 31579 raft_consensus.cc:2468] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:15.719463 32322 leader_election.cc:304] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:15.720469 32558 raft_consensus.cc:2804] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:15.721076 32557 raft_consensus.cc:697] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:15.720957 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "096e1ab4f2704ddaaad535361f46d6a2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:15.720959 32558 raft_consensus.cc:697] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:15.721903 31718 raft_consensus.cc:3060] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.722664 32558 consensus_queue.cc:237] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.732077 31718 raft_consensus.cc:2468] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:15.733455 32557 consensus_queue.cc:237] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.743604 31578 raft_consensus.cc:2468] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.754087 31401 catalog_manager.cc:5697] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 2 to 3, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.781483 31401 catalog_manager.cc:5697] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.822950 32555 raft_consensus.cc:993] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.823448 32555 raft_consensus.cc:1081] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.824872 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ee706eb478f44e0979857553c460e94"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:15.825327 32363 raft_consensus.cc:493] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:15.825639 32363 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.831413 32363 raft_consensus.cc:515] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.833235 32363 leader_election.cc:290] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:15.834306 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:15.835629 31578 raft_consensus.cc:3055] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.835925 31578 raft_consensus.cc:740] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:15.834801 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ee706eb478f44e0979857553c460e94" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:15.838294 31578 consensus_queue.cc:260] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.839221 31718 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.839617 31578 raft_consensus.cc:3060] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.846024 31578 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.845992 31718 raft_consensus.cc:2468] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.847332 32322 leader_election.cc:304] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:15.848142 32557 raft_consensus.cc:2804] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:15.848623 32557 raft_consensus.cc:697] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:15.849476 32557 consensus_queue.cc:237] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:15.862010 31401 catalog_manager.cc:5697] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 2 to 3, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.909504 32543 raft_consensus.cc:993] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.910053 32555 raft_consensus.cc:1081] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.911669 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:15.912191 32363 raft_consensus.cc:493] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:15.912457 32363 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.919627 32363 raft_consensus.cc:515] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.922144 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:15.922719 31578 raft_consensus.cc:3055] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:15.923043 31578 raft_consensus.cc:740] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:15.923446 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae5128ee18fc4cedad5f68bd0e804114" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:15.924079 31651 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.924260 31578 consensus_queue.cc:260] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.924973 31578 raft_consensus.cc:3060] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:15.923188 32363 leader_election.cc:290] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:15.932197 31578 raft_consensus.cc:2468] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:15.933166 32322 leader_election.cc:304] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:15.934042 31651 raft_consensus.cc:2468] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:15.934222 32557 raft_consensus.cc:2804] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:15.934840 31649 raft_consensus.cc:1240] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Rejecting Update request from peer 843512b2aa644b55a21b7e702943e02f for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:15.935055 32557 raft_consensus.cc:697] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:15.936108 32557 consensus_queue.cc:237] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:15.938792 32541 raft_consensus.cc:993] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.939347 32541 raft_consensus.cc:1081] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:15.940810 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:15.941299 32363 raft_consensus.cc:493] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:15.941659 32363 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.948415 32363 raft_consensus.cc:515] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:15.949759 31401 catalog_manager.cc:5697] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:15.950721 32363 leader_election.cc:290] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:15.951056 31651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:15.951637 31651 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.952070 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe911dcdb0624a2293e42d3d26312b21" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:15.952597 31718 raft_consensus.cc:3055] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.952872 31718 raft_consensus.cc:740] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:15.953693 31718 consensus_queue.cc:260] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:15.954777 31718 raft_consensus.cc:3060] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.959283 31651 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.960582 32324 leader_election.cc:304] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:15.961273 32557 raft_consensus.cc:2804] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:15.961799 32557 raft_consensus.cc:697] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:15.962602 32557 consensus_queue.cc:237] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:15.964476 31718 raft_consensus.cc:2468] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:15.976629 31401 catalog_manager.cc:5697] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 2 to 3, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.077340 32556 raft_consensus.cc:993] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:16.077898 32555 raft_consensus.cc:1081] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:16.079403 31651 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:38404
I20260709 12:17:16.079854 31651 raft_consensus.cc:493] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:16.080116 31651 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.087393 31651 raft_consensus.cc:515] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:16.090272 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:16.090411 31651 leader_election.cc:290] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:16.090845 31578 raft_consensus.cc:3055] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.091138 31578 raft_consensus.cc:740] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:16.091394 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ac1bb8ec79464ebaa8b8d0d4a5747d" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:16.091967 31718 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.091892 31578 consensus_queue.cc:260] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:16.093266 31578 raft_consensus.cc:3060] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.098942 31718 raft_consensus.cc:2468] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:16.100106 31616 leader_election.cc:304] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:16.100946 32548 raft_consensus.cc:2804] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:16.101125 31578 raft_consensus.cc:2468] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:16.101380 32548 raft_consensus.cc:697] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:16.102371 32548 consensus_queue.cc:237] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:16.112962 31401 catalog_manager.cc:5697] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.114856 31718 raft_consensus.cc:1275] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.116956 32557 consensus_queue.cc:1048] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.134703 31578 raft_consensus.cc:1275] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.136096 32557 consensus_queue.cc:1048] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.142772 31649 raft_consensus.cc:1275] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:16.149318 32557 consensus_queue.cc:1048] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.176082 31579 raft_consensus.cc:1275] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:16.177848 32558 consensus_queue.cc:1048] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.334676 31649 raft_consensus.cc:1275] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.335995 32558 consensus_queue.cc:1048] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.344957 31579 raft_consensus.cc:1275] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.347143 32557 consensus_queue.cc:1048] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.385047 31649 raft_consensus.cc:1275] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:16.386598 32557 consensus_queue.cc:1048] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:16.392481 31718 raft_consensus.cc:1275] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:16.394214 32557 consensus_queue.cc:1048] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:16.397743 31719 raft_consensus.cc:1275] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:16.399945 32559 consensus_queue.cc:1048] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:16.410411 31579 raft_consensus.cc:1275] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:16.414003 32559 consensus_queue.cc:1048] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:16.484724 31718 raft_consensus.cc:1275] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.485852 32548 consensus_queue.cc:1048] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:16.496999 31578 raft_consensus.cc:1275] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:16.500160 32579 consensus_queue.cc:1048] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:16.623663 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:16.662951 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:16.663483 31578 raft_consensus.cc:606] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:16.666280 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5590853f22ef45de955f77e7648a574b"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:16.666878 31722 raft_consensus.cc:606] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:16.669768 31578 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:16.670334 31578 raft_consensus.cc:606] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:16.673954 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:16.674561 31722 raft_consensus.cc:606] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:16.698310 31649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:17:16.698894 31649 raft_consensus.cc:606] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:16.700570 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:16.702289 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:16.702955 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:16.765681 32541 raft_consensus.cc:993] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:16.766206 32576 raft_consensus.cc:1081] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:16.767607 32370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:16.768023 32370 raft_consensus.cc:493] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:16.768404 32370 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.775079 32370 raft_consensus.cc:515] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:16.775979 32541 raft_consensus.cc:993] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:16.776691 32592 raft_consensus.cc:1081] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:16.778849 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:16.779182 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b667dda9be9d46cdba95860c4c37102e" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:16.779500 31579 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.779817 31718 raft_consensus.cc:3055] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.780175 31718 raft_consensus.cc:740] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:16.781031 31718 consensus_queue.cc:260] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:16.779426 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5590853f22ef45de955f77e7648a574b"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:16.782115 31718 raft_consensus.cc:3060] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.782529 32363 raft_consensus.cc:493] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:16.782924 32363 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.789108 31718 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:16.790539 32321 leader_election.cc:304] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:16.790871 32370 leader_election.cc:290] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:16.791450 32558 raft_consensus.cc:2804] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:16.791965 32558 raft_consensus.cc:697] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:16.792868 32558 consensus_queue.cc:237] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:16.794636 32363 raft_consensus.cc:515] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:16.799181 32363 leader_election.cc:290] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:16.800280 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:16.800662 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5590853f22ef45de955f77e7648a574b" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:16.801030 31722 raft_consensus.cc:3055] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.801302 31650 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.801481 31722 raft_consensus.cc:740] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:16.802388 31722 consensus_queue.cc:260] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:16.803746 31722 raft_consensus.cc:3060] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.807660 31399 catalog_manager.cc:5697] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.808838 31579 raft_consensus.cc:2468] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:16.812970 31722 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:16.813328 31650 raft_consensus.cc:2468] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:16.817525 32321 leader_election.cc:304] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:16.818356 32558 raft_consensus.cc:2804] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:16.818912 32558 raft_consensus.cc:697] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:16.830298 32558 consensus_queue.cc:237] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:16.846205 31399 catalog_manager.cc:5697] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.930573 32556 raft_consensus.cc:993] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:16.932370 32556 raft_consensus.cc:1081] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:16.959125 31650 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:38404
I20260709 12:17:16.960255 31650 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:16.960814 31650 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:16.979522 31650 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:16.991303 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:16.992416 31578 raft_consensus.cc:3055] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:16.992832 31578 raft_consensus.cc:740] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:16.994283 31650 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:16.996457 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:16.997295 31722 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.002580 31578 consensus_queue.cc:260] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:17.004400 31578 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.020702 31722 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:17.022323 31616 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, bf0f149240cc4f27b97810c90eba7ff9; no voters: 
I20260709 12:17:17.022806 31578 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 2.
I20260709 12:17:17.023309 32547 raft_consensus.cc:2804] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.023906 32547 raft_consensus.cc:697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:17.024962 32547 consensus_queue.cc:237] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:17.047030 31397 catalog_manager.cc:5697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 2 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.087661 32579 raft_consensus.cc:993] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.088348 32579 raft_consensus.cc:1081] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.090683 32370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35874
I20260709 12:17:17.091236 32370 raft_consensus.cc:493] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.091651 32370 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.100536 32370 raft_consensus.cc:515] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.103137 32370 leader_election.cc:290] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:17.103744 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:17.104393 31578 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.104487 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f174edb7d6fa49b4a8d02fcdae3fdbe3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:17.105252 31649 raft_consensus.cc:3055] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.105614 31649 raft_consensus.cc:740] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:17.107044 31649 consensus_queue.cc:260] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.109388 31649 raft_consensus.cc:3060] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.113663 31578 raft_consensus.cc:2468] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.115242 32322 leader_election.cc:304] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:17.116775 32558 raft_consensus.cc:2804] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.117345 32558 raft_consensus.cc:697] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:17.118232 31649 raft_consensus.cc:2468] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.119563 32558 consensus_queue.cc:237] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.140374 31397 catalog_manager.cc:5697] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.166311 32555 raft_consensus.cc:993] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.166850 32556 raft_consensus.cc:1081] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.169106 32370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:17.169836 32370 raft_consensus.cc:493] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.170321 32370 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.178755 32370 raft_consensus.cc:515] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.180346 32370 leader_election.cc:290] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:17.181185 31578 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:17.181461 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd06ecf6551463ba3c93bcf334bb6a9" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:17.181960 31649 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.181844 31578 raft_consensus.cc:3055] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.183346 31578 raft_consensus.cc:740] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:17.184212 31578 consensus_queue.cc:260] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.185699 31578 raft_consensus.cc:3060] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.194212 31649 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.195194 31578 raft_consensus.cc:2468] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.195650 32324 leader_election.cc:304] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:17.196996 32558 raft_consensus.cc:2804] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.197542 32558 raft_consensus.cc:697] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:17.198493 32558 consensus_queue.cc:237] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:17.209204 31397 catalog_manager.cc:5697] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.227332 31649 raft_consensus.cc:1275] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.228452 32558 consensus_queue.cc:1048] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.239665 31722 raft_consensus.cc:1275] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.241737 32558 consensus_queue.cc:1048] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:17.420619 31578 raft_consensus.cc:1275] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.421911 31722 raft_consensus.cc:1275] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.422127 32558 consensus_queue.cc:1048] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.424965 32558 consensus_queue.cc:1048] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.595623 31579 raft_consensus.cc:1275] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.596974 32548 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.603309 31650 raft_consensus.cc:1275] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.604835 32585 consensus_queue.cc:1048] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.607451 31722 raft_consensus.cc:1275] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.622421 32547 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:17.622408 31578 raft_consensus.cc:1275] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.626087 32558 consensus_queue.cc:1048] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.649246 31649 raft_consensus.cc:1275] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.650761 32558 consensus_queue.cc:1048] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:17.664708 31578 raft_consensus.cc:1275] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:17.666765 32558 consensus_queue.cc:1048] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:17.743687 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:17.781066 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0281541aef814bce9263539d80488282"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:17.781679 31722 raft_consensus.cc:606] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:17.784386 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:17.784861 31722 raft_consensus.cc:606] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:17.787024 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "544954717af441188c2a2b0ff898f279"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:17.787472 31722 raft_consensus.cc:606] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:17.789710 31649 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
mode: GRACEFUL
new_leader_uuid: "843512b2aa644b55a21b7e702943e02f"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:17:17.790167 31649 raft_consensus.cc:606] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Received request to transfer leadership to TS 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:17.792565 31722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998"
dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
mode: GRACEFUL
new_leader_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:43130
I20260709 12:17:17.793077 31722 raft_consensus.cc:606] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 LEADER]: Received request to transfer leadership to TS 0bd0f6deb3b147509a008c574d9d879d
I20260709 12:17:17.794090 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:17:17.795300 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:17.796234 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:17.960520 32592 raft_consensus.cc:993] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.962064 32541 raft_consensus.cc:1081] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:17.964210 32370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "544954717af441188c2a2b0ff898f279"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:17.964751 32370 raft_consensus.cc:493] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.965088 32370 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.972411 32370 raft_consensus.cc:515] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:17.974313 32370 leader_election.cc:290] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:17.975010 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:17.975651 31650 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.976368 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "544954717af441188c2a2b0ff898f279" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:17.976984 31722 raft_consensus.cc:3055] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.977300 31722 raft_consensus.cc:740] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:17.978155 31722 consensus_queue.cc:260] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:17.979411 31722 raft_consensus.cc:3060] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.986263 31650 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.987453 31722 raft_consensus.cc:2468] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:17.987529 32324 leader_election.cc:304] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:17.988380 32558 raft_consensus.cc:2804] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.988875 32558 raft_consensus.cc:697] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:17.989770 32558 consensus_queue.cc:237] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:18.004230 31399 catalog_manager.cc:5697] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:18.020777 32619 raft_consensus.cc:993] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:18.021266 32592 raft_consensus.cc:1081] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:18.022904 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:18.023422 32363 raft_consensus.cc:493] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:18.023835 32363 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.032306 32363 raft_consensus.cc:515] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:18.034750 32363 leader_election.cc:290] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Requested vote from peers bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:18.036859 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:18.037544 31579 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.040634 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f18f4e3f28a4fa7844c90b02bea7476" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:18.041319 31722 raft_consensus.cc:3055] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.041857 31722 raft_consensus.cc:740] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:18.042716 31722 consensus_queue.cc:260] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:18.044368 31722 raft_consensus.cc:3060] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.051476 31579 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:18.054054 32322 leader_election.cc:304] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:18.052455 32619 raft_consensus.cc:993] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:18.055541 32585 raft_consensus.cc:2804] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:18.055828 32592 raft_consensus.cc:1081] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:18.056007 32585 raft_consensus.cc:697] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:18.057020 32585 consensus_queue.cc:237] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:18.057578 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0281541aef814bce9263539d80488282"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35848
I20260709 12:17:18.061887 31722 raft_consensus.cc:2468] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:18.058399 32363 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:18.070830 32363 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.078248 32363 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:18.080669 32363 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:18.081010 31649 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:18.081692 31649 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.082171 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0281541aef814bce9263539d80488282" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:18.082759 31722 raft_consensus.cc:3055] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:18.083029 31722 raft_consensus.cc:740] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:18.083806 31722 consensus_queue.cc:260] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:18.085611 31722 raft_consensus.cc:3060] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.090615 31649 raft_consensus.cc:2468] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:18.090328 31399 catalog_manager.cc:5697] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 2 to 3, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.091806 32324 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:18.092598 32558 raft_consensus.cc:2804] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:18.093116 32558 raft_consensus.cc:697] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:18.093116 31722 raft_consensus.cc:2468] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:18.094272 32558 consensus_queue.cc:237] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:18.118125 31399 catalog_manager.cc:5697] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:18.183919 32627 raft_consensus.cc:993] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: : Instructing follower 843512b2aa644b55a21b7e702943e02f to start an election
I20260709 12:17:18.185088 32627 raft_consensus.cc:1081] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Signalling peer 843512b2aa644b55a21b7e702943e02f to start an election
I20260709 12:17:18.186164 32620 raft_consensus.cc:993] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: : Instructing follower 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:18.186759 32619 raft_consensus.cc:1081] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 LEADER]: Signalling peer 0bd0f6deb3b147509a008c574d9d879d to start an election
I20260709 12:17:18.186990 31579 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
 from {username='slave'} at 127.0.0.1:56522
I20260709 12:17:18.187551 31579 raft_consensus.cc:493] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:18.187870 31579 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.188391 31649 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
 from {username='slave'} at 127.0.0.1:38388
I20260709 12:17:18.189349 31649 raft_consensus.cc:493] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:18.189728 31649 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:18.195618 31579 raft_consensus.cc:515] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.197608 31579 leader_election.cc:290] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Requested vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:18.197844 31649 raft_consensus.cc:515] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.198185 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:18.198632 31722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71ec83ad93ff4b3abcf4fe9f6005aa7c" candidate_uuid: "843512b2aa644b55a21b7e702943e02f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:18.198714 31650 raft_consensus.cc:3055] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:18.199141 31650 raft_consensus.cc:740] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:18.199122 31722 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.200052 31649 leader_election.cc:290] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 5 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196:40451)
I20260709 12:17:18.200111 31650 consensus_queue.cc:260] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.200645 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:18.202157 31718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a194f9fff2f94f8d8ecfe700ee83d998" candidate_uuid: "0bd0f6deb3b147509a008c574d9d879d" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "bf0f149240cc4f27b97810c90eba7ff9"
I20260709 12:17:18.202308 31579 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:18.202759 31718 raft_consensus.cc:3055] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:18.203174 31718 raft_consensus.cc:740] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: bf0f149240cc4f27b97810c90eba7ff9, State: Running, Role: LEADER
I20260709 12:17:18.203655 31650 raft_consensus.cc:3060] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:18.204008 31718 consensus_queue.cc:260] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.210027 31718 raft_consensus.cc:3060] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:18.211587 31650 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 3.
I20260709 12:17:18.212859 31579 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 5.
I20260709 12:17:18.213356 31545 leader_election.cc:304] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:18.214135 31722 raft_consensus.cc:2468] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 843512b2aa644b55a21b7e702943e02f in term 3.
I20260709 12:17:18.214224 32555 raft_consensus.cc:2804] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:18.214707 31617 leader_election.cc:304] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:18.214793 32555 raft_consensus.cc:697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Becoming Leader. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:18.216630 32588 raft_consensus.cc:2804] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:18.217396 32555 consensus_queue.cc:237] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.220868 31718 raft_consensus.cc:2468] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bd0f6deb3b147509a008c574d9d879d in term 5.
I20260709 12:17:18.222740 32588 raft_consensus.cc:697] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 5 LEADER]: Becoming Leader. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:18.223788 32588 consensus_queue.cc:237] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } }
I20260709 12:17:18.238314 31399 catalog_manager.cc:5697] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d reported cstate change: term changed from 4 to 5, leader changed from bf0f149240cc4f27b97810c90eba7ff9 (127.21.150.196) to 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195). New cstate: current_term: 5 leader_uuid: "0bd0f6deb3b147509a008c574d9d879d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.244148 31397 catalog_manager.cc:5697] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f reported cstate change: term changed from 2 to 3, leader changed from 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195) to 843512b2aa644b55a21b7e702943e02f (127.21.150.194). New cstate: current_term: 3 leader_uuid: "843512b2aa644b55a21b7e702943e02f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:18.451763 31718 raft_consensus.cc:1275] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:18.453327 32557 consensus_queue.cc:1048] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.467593 31578 raft_consensus.cc:1275] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:18.468986 32585 consensus_queue.cc:1048] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.502486 31649 raft_consensus.cc:1275] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.504035 32585 consensus_queue.cc:1048] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.513990 31718 raft_consensus.cc:1275] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.516732 31718 raft_consensus.cc:1275] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.516609 32558 consensus_queue.cc:1048] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.518507 32558 consensus_queue.cc:1048] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.537110 31649 raft_consensus.cc:1275] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.539048 32585 consensus_queue.cc:1048] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.602562 31718 raft_consensus.cc:1275] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:18.603679 32556 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.617162 31649 raft_consensus.cc:1275] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Refusing update from remote peer 843512b2aa644b55a21b7e702943e02f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:18.618851 32555 consensus_queue.cc:1048] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.797271 31718 raft_consensus.cc:1275] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 5 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:18.798496 32588 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:18.812049 31579 raft_consensus.cc:1275] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 5 FOLLOWER]: Refusing update from remote peer 0bd0f6deb3b147509a008c574d9d879d: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:17:18.813920 32588 consensus_queue.cc:1048] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:18.835676 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:18.874635 31650 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3"
dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:59726
I20260709 12:17:18.875250 31650 raft_consensus.cc:606] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:18.877785 31579 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2"
dest_uuid: "843512b2aa644b55a21b7e702943e02f"
mode: GRACEFUL
new_leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:57908
I20260709 12:17:18.878356 31579 raft_consensus.cc:606] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Received request to transfer leadership to TS 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:18.879642 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:18.881175 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:18.881916 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:19.094398 32556 raft_consensus.cc:993] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:19.094938 32555 raft_consensus.cc:1081] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:19.096323 32370 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35860
I20260709 12:17:19.096840 32370 raft_consensus.cc:493] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:19.097174 32370 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.103081 32370 raft_consensus.cc:515] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.104717 32370 leader_election.cc:290] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:19.105337 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:19.105722 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeff768d3659498c9e632d68b32a28d2" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:19.106003 31579 raft_consensus.cc:3055] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:19.106288 31650 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.106314 31579 raft_consensus.cc:740] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 843512b2aa644b55a21b7e702943e02f, State: Running, Role: LEADER
I20260709 12:17:19.107303 31579 consensus_queue.cc:260] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.108491 31579 raft_consensus.cc:3060] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:19.113346 31650 raft_consensus.cc:2468] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:19.114549 32324 leader_election.cc:304] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4; no voters: 
I20260709 12:17:19.115141 31579 raft_consensus.cc:2468] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 2.
I20260709 12:17:19.115213 32558 raft_consensus.cc:2804] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:19.115783 32558 raft_consensus.cc:697] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:19.116622 32558 consensus_queue.cc:237] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.126955 31399 catalog_manager.cc:5697] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 1 to 2, leader changed from 843512b2aa644b55a21b7e702943e02f (127.21.150.194) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.392900 32628 raft_consensus.cc:993] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: : Instructing follower 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:19.393345 32644 raft_consensus.cc:1081] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Signalling peer 7dd13c0178c84f7aaab5c435a8b205e4 to start an election
I20260709 12:17:19.395195 32363 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3"
dest_uuid: "7dd13c0178c84f7aaab5c435a8b205e4"
 from {username='slave'} at 127.0.0.1:35874
I20260709 12:17:19.395726 32363 raft_consensus.cc:493] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:19.395959 32363 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.402267 32363 raft_consensus.cc:515] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.404260 32363 leader_election.cc:290] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Requested vote from peers 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965), 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461)
I20260709 12:17:19.404611 31579 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "843512b2aa644b55a21b7e702943e02f"
I20260709 12:17:19.404935 31650 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1937adc6d4be49a0aa4d15f000842da3" candidate_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bd0f6deb3b147509a008c574d9d879d"
I20260709 12:17:19.405208 31579 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.405668 31650 raft_consensus.cc:3055] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:19.405982 31650 raft_consensus.cc:740] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bd0f6deb3b147509a008c574d9d879d, State: Running, Role: LEADER
I20260709 12:17:19.406666 31650 consensus_queue.cc:260] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.407693 31650 raft_consensus.cc:3060] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:19.410964 31579 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:19.412135 32322 leader_election.cc:304] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f; no voters: 
I20260709 12:17:19.413116 32585 raft_consensus.cc:2804] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:19.413605 32585 raft_consensus.cc:697] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Becoming Leader. State: Replica: 7dd13c0178c84f7aaab5c435a8b205e4, State: Running, Role: LEADER
I20260709 12:17:19.414291 31650 raft_consensus.cc:2468] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd13c0178c84f7aaab5c435a8b205e4 in term 3.
I20260709 12:17:19.414822 32585 consensus_queue.cc:237] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } }
I20260709 12:17:19.425031 31399 catalog_manager.cc:5697] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 reported cstate change: term changed from 2 to 3, leader changed from 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195) to 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193). New cstate: current_term: 3 leader_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:19.603974 31579 raft_consensus.cc:1275] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:19.605362 32558 consensus_queue.cc:1048] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:19.624544 31650 raft_consensus.cc:1275] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:19.625841 32558 consensus_queue.cc:1048] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:19.830197 31650 raft_consensus.cc:1275] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:19.831341 32557 consensus_queue.cc:1048] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:19.840154 31578 raft_consensus.cc:1275] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Refusing update from remote peer 7dd13c0178c84f7aaab5c435a8b205e4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:19.841627 32558 consensus_queue.cc:1048] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:19.919638 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:19.956046 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:19.957136 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:19.957743 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:19.999696 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:20.040237 22107 tablet_server.cc:179] TabletServer@127.21.150.193:37911 shutting down...
W20260709 12:17:20.046478 31690 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 94 similar messages]
W20260709 12:17:20.059360 31690 consensus_peers.cc:597] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.085726 31690 consensus_peers.cc:597] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.090353 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:20.091944 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.093195 22107 raft_consensus.cc:2243] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.093806 22107 raft_consensus.cc:2272] T 78d7a21102b64e7f95625a42c84e9ad0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.096613 31542 consensus_peers.cc:597] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.097633 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.098553 22107 raft_consensus.cc:2243] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.099162 22107 raft_consensus.cc:2272] T 098e97adee224f86ba1bfa07ce392b68 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.102833 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.104225 22107 raft_consensus.cc:2243] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:20.104696 31542 consensus_peers.cc:597] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.104840 22107 raft_consensus.cc:2272] T af22442d6a9141f2a4607ba57cf4e0ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.108188 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.108848 22107 raft_consensus.cc:2243] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.109408 22107 raft_consensus.cc:2272] T 6d00a005dd8945d9acac25d9822223a6 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.111735 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.112406 22107 raft_consensus.cc:2243] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.112816 22107 raft_consensus.cc:2272] T ecff280f5652462db2e6473ea7424a2f P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.115345 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.116202 22107 raft_consensus.cc:2243] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.116732 22107 raft_consensus.cc:2272] T 68cd2a3bb89a49a6baa7af253a090766 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.119488 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.120175 22107 raft_consensus.cc:2243] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.120639 22107 raft_consensus.cc:2272] T c506263e3b0d4ce98f8336aec8159188 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.123565 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.124610 22107 raft_consensus.cc:2243] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.125638 22107 raft_consensus.cc:2272] T 1937adc6d4be49a0aa4d15f000842da3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.127310 31616 consensus_peers.cc:597] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.128885 31616 consensus_peers.cc:597] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.129098 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.129859 22107 raft_consensus.cc:2243] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.130303 22107 raft_consensus.cc:2272] T e6c8433c534b4d19bc10a5cb0ca094ba P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.133415 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.134356 22107 raft_consensus.cc:2243] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.134984 22107 raft_consensus.cc:2272] T 6019e17e78a14e9fa70a4f5aa6e26056 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.137660 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.138576 22107 raft_consensus.cc:2243] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.139571 22107 raft_consensus.cc:2272] T 3f18f4e3f28a4fa7844c90b02bea7476 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.142212 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.142815 22107 raft_consensus.cc:2243] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.143807 22107 raft_consensus.cc:2272] T e4ba9fa8e8004145a227359512e2443d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.145987 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.146553 22107 raft_consensus.cc:2243] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.147341 22107 raft_consensus.cc:2272] T fe911dcdb0624a2293e42d3d26312b21 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.149578 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.150341 22107 raft_consensus.cc:2243] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.150904 22107 raft_consensus.cc:2272] T e5dfd0d3d6f941e68232bcc32722b5de P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.153532 31616 consensus_peers.cc:597] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.153818 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.154810 22107 raft_consensus.cc:2243] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.155337 22107 raft_consensus.cc:2272] T 59dc1ab82c334d8f8954116a598b7c8b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.158577 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.159498 22107 raft_consensus.cc:2243] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.160470 22107 raft_consensus.cc:2272] T 544954717af441188c2a2b0ff898f279 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.163933 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.164731 22107 raft_consensus.cc:2243] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.165159 22107 raft_consensus.cc:2272] T c130f9abd9da4eaab759d4a5de2c1e86 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.167338 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.168006 22107 raft_consensus.cc:2243] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.168771 22107 raft_consensus.cc:2272] T aeff768d3659498c9e632d68b32a28d2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.171393 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.172220 22107 raft_consensus.cc:2243] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.172729 22107 raft_consensus.cc:2272] T 384f88a7446842bfb4a6c71e921c9049 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.175915 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.176673 22107 raft_consensus.cc:2243] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.177234 22107 raft_consensus.cc:2272] T 1f6baf18d4c74681ab192824db201eec P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.178694 31616 consensus_peers.cc:597] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.179914 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.180583 22107 raft_consensus.cc:2243] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.181031 22107 raft_consensus.cc:2272] T dc169af544e14e668b85ec4da6ca648d P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.183725 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.184438 22107 raft_consensus.cc:2243] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.185122 22107 raft_consensus.cc:2272] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.187083 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.187573 22107 raft_consensus.cc:2243] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.188275 22107 raft_consensus.cc:2272] T 096e1ab4f2704ddaaad535361f46d6a2 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.190265 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.190799 22107 raft_consensus.cc:2243] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.191336 22107 raft_consensus.cc:2272] T 28ff8e905a7b4eb5b2cea1df313efa14 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.193820 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.194414 22107 raft_consensus.cc:2243] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.194849 22107 raft_consensus.cc:2272] T 4dfd13293d3149f6a1ca73fd10852083 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.196949 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.197671 22107 raft_consensus.cc:2243] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.198119 22107 raft_consensus.cc:2272] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.200639 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.201460 22107 raft_consensus.cc:2243] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.202611 22107 raft_consensus.cc:2272] T b667dda9be9d46cdba95860c4c37102e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.205475 22107 tablet_replica.cc:333] stopping tablet replica
W20260709 12:17:20.205673 31542 consensus_peers.cc:597] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.206234 22107 raft_consensus.cc:2243] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.207263 22107 raft_consensus.cc:2272] T 5590853f22ef45de955f77e7648a574b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.210420 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.211313 22107 raft_consensus.cc:2243] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.212325 22107 raft_consensus.cc:2272] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.214967 31616 consensus_peers.cc:597] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.215281 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.215916 22107 raft_consensus.cc:2243] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.216229 22107 raft_consensus.cc:2272] T fd72db62f85b4234aee25b5eda5abddd P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.218267 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.218767 22107 raft_consensus.cc:2243] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.219451 22107 raft_consensus.cc:2272] T 0281541aef814bce9263539d80488282 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.221407 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.221997 22107 raft_consensus.cc:2243] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.222344 22107 raft_consensus.cc:2272] T a2cd62a9ae5d4430bdb3799b63ca2137 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.224258 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.224839 22107 raft_consensus.cc:2243] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.225378 22107 raft_consensus.cc:2272] T 9da10e70d33e4495babdca0057997818 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.228274 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.229173 22107 raft_consensus.cc:2243] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.229702 22107 raft_consensus.cc:2272] T 506b4da2b0be46d5bf3ff52167d4419b P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.232652 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.233377 22107 raft_consensus.cc:2243] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.233955 22107 raft_consensus.cc:2272] T b76fc75555e64cd3a5762910676fb4e3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.236267 31690 consensus_peers.cc:597] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.236932 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.237555 22107 raft_consensus.cc:2243] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.237975 22107 raft_consensus.cc:2272] T dc5b5e283ec34f03b184be6dfa4e046e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.240338 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.241063 22107 raft_consensus.cc:2243] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.242067 22107 raft_consensus.cc:2272] T ae5128ee18fc4cedad5f68bd0e804114 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.244618 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.245223 22107 raft_consensus.cc:2243] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.245865 22107 raft_consensus.cc:2272] T 7a5f0372a71a4abe9b70c3772934163c P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.248569 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.249239 22107 raft_consensus.cc:2243] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.249683 22107 raft_consensus.cc:2272] T 53b9a430b2d54764b10115eae2e0e653 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.252177 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.252673 22107 raft_consensus.cc:2243] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.253129 22107 raft_consensus.cc:2272] T 33acdf64b87145ab8a38a66147ae23b5 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.255308 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.255877 22107 raft_consensus.cc:2243] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.256320 22107 raft_consensus.cc:2272] T 6ad0edffb3874e40994f98fd888a00b3 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.258793 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.259472 22107 raft_consensus.cc:2243] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.260016 22107 raft_consensus.cc:2272] T abd9c92a3ff84f17a24aeceaffae4ef9 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.261564 31542 consensus_peers.cc:597] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.262691 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.263221 22107 raft_consensus.cc:2243] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.263881 22107 raft_consensus.cc:2272] T 9ee706eb478f44e0979857553c460e94 P 7dd13c0178c84f7aaab5c435a8b205e4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.266588 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.267283 22107 raft_consensus.cc:2243] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.267793 22107 raft_consensus.cc:2272] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.270674 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:20.271266 22107 raft_consensus.cc:2243] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.271761 22107 raft_consensus.cc:2272] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 7dd13c0178c84f7aaab5c435a8b205e4 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.273797 31542 consensus_peers.cc:597] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.290212 31690 consensus_peers.cc:597] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.301697 31690 consensus_peers.cc:597] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.321637 31616 consensus_peers.cc:597] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.325676 31616 consensus_peers.cc:597] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.341552 22107 tablet_server.cc:196] TabletServer@127.21.150.193:37911 shutdown complete.
W20260709 12:17:20.364832 31690 consensus_peers.cc:597] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.419562 31542 consensus_peers.cc:597] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.425664 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:17:20.428912 31542 consensus_peers.cc:597] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.429859 31690 consensus_peers.cc:597] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.434037 31690 consensus_peers.cc:597] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.459318 31617 consensus_peers.cc:597] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.463259 31542 consensus_peers.cc:597] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.469291 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:20.470432 22107 tablet_replica.cc:333] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.471457 22107 raft_consensus.cc:2243] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.472223 22107 raft_consensus.cc:2272] T 2d3dff40fad349cd90ec48c6ccc15f20 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.475378 22107 tablet_replica.cc:333] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.476181 22107 raft_consensus.cc:2243] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.477190 22107 raft_consensus.cc:2272] T e6c8433c534b4d19bc10a5cb0ca094ba P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.478063 31542 consensus_peers.cc:597] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.480214 22107 tablet_replica.cc:333] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.481192 22107 raft_consensus.cc:2243] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.482210 22107 raft_consensus.cc:2272] T 68cd2a3bb89a49a6baa7af253a090766 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.485193 22107 tablet_replica.cc:333] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.485881 22107 raft_consensus.cc:2243] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.486344 22107 raft_consensus.cc:2272] T e302635e586c48819cf549549c5f4446 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.488447 22107 tablet_replica.cc:333] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.489367 22107 raft_consensus.cc:2243] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.489944 22107 raft_consensus.cc:2272] T 78d7a21102b64e7f95625a42c84e9ad0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.492995 22107 tablet_replica.cc:333] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.493886 22107 raft_consensus.cc:2243] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:20.493971 31542 consensus_peers.cc:597] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.494596 22107 raft_consensus.cc:2272] T ae5128ee18fc4cedad5f68bd0e804114 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.496977 22107 tablet_replica.cc:333] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.497805 22107 raft_consensus.cc:2243] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.498831 22107 raft_consensus.cc:2272] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.501622 22107 tablet_replica.cc:333] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.502239 22107 raft_consensus.cc:2243] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.503039 22107 raft_consensus.cc:2272] T 9da10e70d33e4495babdca0057997818 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.505195 22107 tablet_replica.cc:333] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.505869 22107 raft_consensus.cc:2243] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.506861 22107 raft_consensus.cc:2272] T 384f88a7446842bfb4a6c71e921c9049 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.509637 22107 tablet_replica.cc:333] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
W20260709 12:17:20.509740 31617 consensus_peers.cc:597] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.510350 22107 raft_consensus.cc:2243] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.511122 22107 raft_consensus.cc:2272] T 1937adc6d4be49a0aa4d15f000842da3 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.517320 22107 tablet_replica.cc:333] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.518257 22107 raft_consensus.cc:2243] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.518884 22107 raft_consensus.cc:2272] T 6ad0edffb3874e40994f98fd888a00b3 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.521776 31617 consensus_peers.cc:597] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.522836 22107 tablet_replica.cc:333] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.523998 22107 raft_consensus.cc:2243] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.524761 22107 raft_consensus.cc:2272] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.528014 22107 tablet_replica.cc:333] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.528891 22107 raft_consensus.cc:2243] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.529824 22107 raft_consensus.cc:2272] T c130f9abd9da4eaab759d4a5de2c1e86 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.533265 22107 tablet_replica.cc:333] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.534258 22107 raft_consensus.cc:2243] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.534827 22107 raft_consensus.cc:2272] T aeff768d3659498c9e632d68b32a28d2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.537796 22107 tablet_replica.cc:333] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.538642 22107 raft_consensus.cc:2243] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.539232 22107 raft_consensus.cc:2272] T 3c241197f69848f4b2142121633ec9ef P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.539287 31690 consensus_peers.cc:597] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.540333 31616 consensus_peers.cc:597] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.542006 22107 tablet_replica.cc:333] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
W20260709 12:17:20.541965 31617 consensus_peers.cc:597] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.543069 22107 raft_consensus.cc:2243] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.543715 22107 raft_consensus.cc:2272] T dbfef93226df40f3a8d07af88af622fb P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.546970 22107 tablet_replica.cc:333] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.547782 22107 raft_consensus.cc:2243] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.548303 22107 raft_consensus.cc:2272] T e5dfd0d3d6f941e68232bcc32722b5de P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.549889 31690 consensus_peers.cc:597] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.551074 22107 tablet_replica.cc:333] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.551733 22107 raft_consensus.cc:2243] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.552536 22107 raft_consensus.cc:2272] T 59dc1ab82c334d8f8954116a598b7c8b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.554697 22107 tablet_replica.cc:333] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.555259 22107 raft_consensus.cc:2243] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.555992 22107 raft_consensus.cc:2272] T 81a98a7e3a4c46e9a01e39bb9924afc0 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.557935 22107 tablet_replica.cc:333] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.558516 22107 raft_consensus.cc:2243] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.558908 22107 raft_consensus.cc:2272] T a44cca173ddc49d99e53c40bccf8b5c9 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.560882 22107 tablet_replica.cc:333] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.561422 22107 raft_consensus.cc:2243] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.561908 22107 raft_consensus.cc:2272] T a194f9fff2f94f8d8ecfe700ee83d998 P 843512b2aa644b55a21b7e702943e02f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.563614 22107 tablet_replica.cc:333] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.564180 22107 raft_consensus.cc:2243] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.564617 22107 raft_consensus.cc:2272] T b667dda9be9d46cdba95860c4c37102e P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.566552 22107 tablet_replica.cc:333] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.567101 22107 raft_consensus.cc:2243] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.567895 22107 raft_consensus.cc:2272] T 4dfd13293d3149f6a1ca73fd10852083 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.569799 22107 tablet_replica.cc:333] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.570317 22107 raft_consensus.cc:2243] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.570735 22107 raft_consensus.cc:2272] T a6c957e6781843b9b69e2d98053daa74 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.572522 22107 tablet_replica.cc:333] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.573041 22107 raft_consensus.cc:2243] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.573860 22107 raft_consensus.cc:2272] T ecff280f5652462db2e6473ea7424a2f P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.575594 22107 tablet_replica.cc:333] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.576115 22107 raft_consensus.cc:2243] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.576541 22107 raft_consensus.cc:2272] T d85a31e6ff114d30b4ae3a4453403a01 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.578753 22107 tablet_replica.cc:333] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.579294 22107 raft_consensus.cc:2243] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.579855 22107 raft_consensus.cc:2272] T e4ba9fa8e8004145a227359512e2443d P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.582235 31691 consensus_peers.cc:597] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.582404 22107 tablet_replica.cc:333] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.583298 22107 raft_consensus.cc:2243] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.583727 22107 raft_consensus.cc:2272] T 506b4da2b0be46d5bf3ff52167d4419b P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.585716 22107 tablet_replica.cc:333] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.586274 22107 raft_consensus.cc:2243] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.587023 22107 raft_consensus.cc:2272] T 75b15073c4e54ef1af602f9c83702642 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.589545 22107 tablet_replica.cc:333] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.590281 22107 raft_consensus.cc:2243] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.591363 22107 raft_consensus.cc:2272] T 53b9a430b2d54764b10115eae2e0e653 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.593405 22107 tablet_replica.cc:333] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.593957 22107 raft_consensus.cc:2243] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.594708 22107 raft_consensus.cc:2272] T 7a5f0372a71a4abe9b70c3772934163c P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.597183 22107 tablet_replica.cc:333] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.598023 22107 raft_consensus.cc:2243] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.598583 22107 raft_consensus.cc:2272] T 33acdf64b87145ab8a38a66147ae23b5 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.599509 31691 consensus_peers.cc:597] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.601387 22107 tablet_replica.cc:333] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.602047 22107 raft_consensus.cc:2243] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.602557 22107 raft_consensus.cc:2272] T 74bd4922c2764afd8737751cb3d17cb8 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.605311 22107 tablet_replica.cc:333] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.605918 22107 raft_consensus.cc:2243] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.606328 22107 raft_consensus.cc:2272] T 45a8924a0b9c4dbc9ace49508d7b9f7e P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.608551 22107 tablet_replica.cc:333] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.609225 22107 raft_consensus.cc:2243] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.609678 22107 raft_consensus.cc:2272] T abd9c92a3ff84f17a24aeceaffae4ef9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.611557 22107 tablet_replica.cc:333] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.612112 22107 raft_consensus.cc:2243] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.612646 22107 raft_consensus.cc:2272] T 9ee706eb478f44e0979857553c460e94 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.615599 22107 tablet_replica.cc:333] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.616304 22107 raft_consensus.cc:2243] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.616842 22107 raft_consensus.cc:2272] T c506263e3b0d4ce98f8336aec8159188 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.619516 22107 tablet_replica.cc:333] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.620244 22107 raft_consensus.cc:2243] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.621263 22107 raft_consensus.cc:2272] T 6019e17e78a14e9fa70a4f5aa6e26056 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.623942 22107 tablet_replica.cc:333] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
W20260709 12:17:20.624408 31617 consensus_peers.cc:597] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.624656 22107 raft_consensus.cc:2243] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:20.625206 31690 consensus_peers.cc:597] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Couldn't send request to peer 7dd13c0178c84f7aaab5c435a8b205e4. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.625237 22107 raft_consensus.cc:2272] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.627415 22107 tablet_replica.cc:333] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.627928 22107 raft_consensus.cc:2243] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.628328 22107 raft_consensus.cc:2272] T 3f18f4e3f28a4fa7844c90b02bea7476 P 843512b2aa644b55a21b7e702943e02f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.630059 22107 tablet_replica.cc:333] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.630549 22107 raft_consensus.cc:2243] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.631285 22107 raft_consensus.cc:2272] T 8e0f4df696784a248aaa0b98db69dfa0 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.633139 22107 tablet_replica.cc:333] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.633693 22107 raft_consensus.cc:2243] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.634122 22107 raft_consensus.cc:2272] T b52b6a2957cc45ae96342d2b42cf8ae1 P 843512b2aa644b55a21b7e702943e02f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.636210 22107 tablet_replica.cc:333] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.636683 22107 raft_consensus.cc:2243] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.637090 22107 raft_consensus.cc:2272] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.638994 22107 tablet_replica.cc:333] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f: stopping tablet replica
I20260709 12:17:20.639444 22107 raft_consensus.cc:2243] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.639838 22107 raft_consensus.cc:2272] T 096e1ab4f2704ddaaad535361f46d6a2 P 843512b2aa644b55a21b7e702943e02f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.677703 31691 consensus_peers.cc:597] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.704478 31691 consensus_peers.cc:597] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.712133 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
W20260709 12:17:20.727367 31617 consensus_peers.cc:597] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.750104 31617 consensus_peers.cc:597] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.750711 31617 consensus_peers.cc:597] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.788774 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
W20260709 12:17:20.798938 31693 consensus_peers.cc:597] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.804307 31617 consensus_peers.cc:597] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.813692 31691 consensus_peers.cc:597] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:20.843341 31691 consensus_peers.cc:597] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.852092 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:20.853124 22107 tablet_replica.cc:333] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.854135 22107 raft_consensus.cc:2243] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.855360 22107 raft_consensus.cc:2272] T fe911dcdb0624a2293e42d3d26312b21 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.855458 31617 consensus_peers.cc:597] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.858124 22107 tablet_replica.cc:333] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.858989 22107 raft_consensus.cc:2243] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.859560 22107 raft_consensus.cc:2272] T 2d3dff40fad349cd90ec48c6ccc15f20 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.862262 22107 tablet_replica.cc:333] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.863008 22107 raft_consensus.cc:2243] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.863425 22107 raft_consensus.cc:2272] T 68cd2a3bb89a49a6baa7af253a090766 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.866003 22107 tablet_replica.cc:333] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.866771 22107 raft_consensus.cc:2243] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.867736 22107 raft_consensus.cc:2272] T abd9c92a3ff84f17a24aeceaffae4ef9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.869750 22107 tablet_replica.cc:333] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.870282 22107 raft_consensus.cc:2243] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.870683 22107 raft_consensus.cc:2272] T ae5128ee18fc4cedad5f68bd0e804114 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.872668 22107 tablet_replica.cc:333] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.873169 22107 raft_consensus.cc:2243] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.873716 22107 raft_consensus.cc:2272] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.875988 22107 tablet_replica.cc:333] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.876876 22107 raft_consensus.cc:2243] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.877955 22107 raft_consensus.cc:2272] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.879370 31617 consensus_peers.cc:597] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.880749 22107 tablet_replica.cc:333] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.881486 22107 raft_consensus.cc:2243] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:20.881628 31617 consensus_peers.cc:597] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.882040 22107 raft_consensus.cc:2272] T 1937adc6d4be49a0aa4d15f000842da3 P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.884047 22107 tablet_replica.cc:333] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.884662 22107 raft_consensus.cc:2243] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.885107 22107 raft_consensus.cc:2272] T 9da10e70d33e4495babdca0057997818 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.887082 22107 tablet_replica.cc:333] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.887688 22107 raft_consensus.cc:2243] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.888497 22107 raft_consensus.cc:2272] T a2cd62a9ae5d4430bdb3799b63ca2137 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.890800 22107 tablet_replica.cc:333] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.891435 22107 raft_consensus.cc:2243] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.891888 22107 raft_consensus.cc:2272] T e302635e586c48819cf549549c5f4446 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.893826 22107 tablet_replica.cc:333] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.894407 22107 raft_consensus.cc:2243] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.895630 22107 raft_consensus.cc:2272] T 3c241197f69848f4b2142121633ec9ef P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.898358 22107 tablet_replica.cc:333] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.898995 22107 raft_consensus.cc:2243] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.899466 22107 raft_consensus.cc:2272] T 5590853f22ef45de955f77e7648a574b P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.901815 22107 tablet_replica.cc:333] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.902504 22107 raft_consensus.cc:2243] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.903028 22107 raft_consensus.cc:2272] T 544954717af441188c2a2b0ff898f279 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.905303 31691 consensus_peers.cc:597] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Couldn't send request to peer 843512b2aa644b55a21b7e702943e02f. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.905617 22107 tablet_replica.cc:333] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.906489 22107 raft_consensus.cc:2243] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.906987 22107 raft_consensus.cc:2272] T c130f9abd9da4eaab759d4a5de2c1e86 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.909660 22107 tablet_replica.cc:333] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.910240 22107 raft_consensus.cc:2243] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.910692 22107 raft_consensus.cc:2272] T aeff768d3659498c9e632d68b32a28d2 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.913503 22107 tablet_replica.cc:333] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.914301 22107 raft_consensus.cc:2243] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.915560 22107 raft_consensus.cc:2272] T 6ad0edffb3874e40994f98fd888a00b3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:20.917543 31693 consensus_peers.cc:597] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.918345 22107 tablet_replica.cc:333] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.919001 22107 raft_consensus.cc:2243] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.919452 22107 raft_consensus.cc:2272] T a6c957e6781843b9b69e2d98053daa74 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.921613 22107 tablet_replica.cc:333] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.922217 22107 raft_consensus.cc:2243] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.922652 22107 raft_consensus.cc:2272] T 6d00a005dd8945d9acac25d9822223a6 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.924639 22107 tablet_replica.cc:333] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.925240 22107 raft_consensus.cc:2243] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.925689 22107 raft_consensus.cc:2272] T 1f6baf18d4c74681ab192824db201eec P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.927690 22107 tablet_replica.cc:333] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.928262 22107 raft_consensus.cc:2243] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.928689 22107 raft_consensus.cc:2272] T f174edb7d6fa49b4a8d02fcdae3fdbe3 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.930789 22107 tablet_replica.cc:333] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.931404 22107 raft_consensus.cc:2243] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.932199 22107 raft_consensus.cc:2272] T dc169af544e14e668b85ec4da6ca648d P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.934141 22107 tablet_replica.cc:333] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.934840 22107 raft_consensus.cc:2243] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.935994 22107 raft_consensus.cc:2272] T 4c6ef1dbe6184d41b2a5edfb45c8315e P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.938886 22107 tablet_replica.cc:333] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.939705 22107 raft_consensus.cc:2243] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.940624 22107 raft_consensus.cc:2272] T d85a31e6ff114d30b4ae3a4453403a01 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.942646 22107 tablet_replica.cc:333] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.943271 22107 raft_consensus.cc:2243] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.944116 22107 raft_consensus.cc:2272] T e4ba9fa8e8004145a227359512e2443d P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.946650 22107 tablet_replica.cc:333] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.947288 22107 raft_consensus.cc:2243] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.947743 22107 raft_consensus.cc:2272] T b76fc75555e64cd3a5762910676fb4e3 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.949852 22107 tablet_replica.cc:333] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.950416 22107 raft_consensus.cc:2243] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.950845 22107 raft_consensus.cc:2272] T 75b15073c4e54ef1af602f9c83702642 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.953104 22107 tablet_replica.cc:333] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.953753 22107 raft_consensus.cc:2243] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.954571 22107 raft_consensus.cc:2272] T 33acdf64b87145ab8a38a66147ae23b5 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.956808 22107 tablet_replica.cc:333] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.957368 22107 raft_consensus.cc:2243] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.957885 22107 raft_consensus.cc:2272] T 7a5f0372a71a4abe9b70c3772934163c P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.959954 22107 tablet_replica.cc:333] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.960508 22107 raft_consensus.cc:2243] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.960929 22107 raft_consensus.cc:2272] T 74bd4922c2764afd8737751cb3d17cb8 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.964361 22107 tablet_replica.cc:333] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
W20260709 12:17:20.964849 31693 consensus_peers.cc:597] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:20.965010 22107 raft_consensus.cc:2243] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.965955 22107 raft_consensus.cc:2272] T c506263e3b0d4ce98f8336aec8159188 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.967922 22107 tablet_replica.cc:333] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.968489 22107 raft_consensus.cc:2243] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.969278 22107 raft_consensus.cc:2272] T a44cca173ddc49d99e53c40bccf8b5c9 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.971175 22107 tablet_replica.cc:333] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.971846 22107 raft_consensus.cc:2243] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:17:20.972776 22107 raft_consensus.cc:2272] T a194f9fff2f94f8d8ecfe700ee83d998 P 0bd0f6deb3b147509a008c574d9d879d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.975242 22107 tablet_replica.cc:333] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.975960 22107 raft_consensus.cc:2243] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.976593 22107 raft_consensus.cc:2272] T 28ff8e905a7b4eb5b2cea1df313efa14 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.978998 22107 tablet_replica.cc:333] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.979841 22107 raft_consensus.cc:2243] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.980422 22107 raft_consensus.cc:2272] T 098e97adee224f86ba1bfa07ce392b68 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.982807 22107 tablet_replica.cc:333] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.983346 22107 raft_consensus.cc:2243] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.983759 22107 raft_consensus.cc:2272] T af22442d6a9141f2a4607ba57cf4e0ba P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.985719 22107 tablet_replica.cc:333] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.986245 22107 raft_consensus.cc:2243] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.986667 22107 raft_consensus.cc:2272] T 0281541aef814bce9263539d80488282 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.989503 22107 tablet_replica.cc:333] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.990242 22107 raft_consensus.cc:2243] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.990782 22107 raft_consensus.cc:2272] T fd72db62f85b4234aee25b5eda5abddd P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.994930 22107 tablet_replica.cc:333] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:20.995823 22107 raft_consensus.cc:2243] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:20.996356 22107 raft_consensus.cc:2272] T 3bd06ecf6551463ba3c93bcf334bb6a9 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:20.999267 22107 tablet_replica.cc:333] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:21.000041 22107 raft_consensus.cc:2243] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:17:21.000650 31693 consensus_peers.cc:597] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:21.001065 22107 raft_consensus.cc:2272] T dbfef93226df40f3a8d07af88af622fb P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:21.012576 31693 consensus_peers.cc:597] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:21.013135 22107 tablet_replica.cc:333] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:21.014003 22107 raft_consensus.cc:2243] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.015055 22107 raft_consensus.cc:2272] T e5dfd0d3d6f941e68232bcc32722b5de P 0bd0f6deb3b147509a008c574d9d879d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.017470 22107 tablet_replica.cc:333] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:21.018049 22107 raft_consensus.cc:2243] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.018501 22107 raft_consensus.cc:2272] T dc5b5e283ec34f03b184be6dfa4e046e P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.020342 22107 tablet_replica.cc:333] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:21.020841 22107 raft_consensus.cc:2243] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.021301 22107 raft_consensus.cc:2272] T 8e0f4df696784a248aaa0b98db69dfa0 P 0bd0f6deb3b147509a008c574d9d879d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.023412 22107 tablet_replica.cc:333] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d: stopping tablet replica
I20260709 12:17:21.023903 22107 raft_consensus.cc:2243] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.024761 22107 raft_consensus.cc:2272] T b52b6a2957cc45ae96342d2b42cf8ae1 P 0bd0f6deb3b147509a008c574d9d879d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:21.035501 31693 consensus_peers.cc:597] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:21.062549 31693 consensus_peers.cc:597] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:21.097281 31618 connection.cc:466] server connection from 127.0.0.1:38388 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38388 (request call id 762) could send its response
W20260709 12:17:21.098788 31693 consensus_peers.cc:597] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Network error: recv got EOF from 127.21.150.195:38461 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:21.114934 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
W20260709 12:17:21.126494 31693 consensus_peers.cc:597] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:21.131757 31693 consensus_peers.cc:597] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:21.134886 31693 consensus_peers.cc:597] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:21.194306 22107 tablet_server.cc:179] TabletServer@127.21.150.196:0 shutting down...
I20260709 12:17:21.236426 32664 raft_consensus.cc:493] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:21.237128 32664 raft_consensus.cc:515] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:21.239488 32664 leader_election.cc:290] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
W20260709 12:17:21.246317 31690 leader_election.cc:336] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
W20260709 12:17:21.248698 31693 leader_election.cc:336] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111)
I20260709 12:17:21.249076 31693 leader_election.cc:304] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:21.249842 32671 raft_consensus.cc:2749] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.251111 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:21.252038 22107 tablet_replica.cc:333] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.252144 32671 raft_consensus.cc:493] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:21.252696 32671 raft_consensus.cc:515] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "843512b2aa644b55a21b7e702943e02f" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 39965 } }
I20260709 12:17:21.253104 22107 raft_consensus.cc:2243] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.254320 22107 raft_consensus.cc:2272] T 2d3dff40fad349cd90ec48c6ccc15f20 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.255463 32671 leader_election.cc:290] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911), 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965)
I20260709 12:17:21.257200 22107 tablet_replica.cc:333] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.258131 22107 raft_consensus.cc:2243] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.258627 22107 raft_consensus.cc:2272] T d85a31e6ff114d30b4ae3a4453403a01 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:21.260093 31693 consensus_peers.cc:597] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 -> Peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Couldn't send request to peer 0bd0f6deb3b147509a008c574d9d879d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:21.260213 31691 leader_election.cc:336] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 843512b2aa644b55a21b7e702943e02f (127.21.150.194:39965): Network error: Client connection negotiation failed: client connection to 127.21.150.194:39965: connect: Connection refused (error 111)
W20260709 12:17:21.260854 31690 leader_election.cc:336] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:21.261197 31690 leader_election.cc:304] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 7dd13c0178c84f7aaab5c435a8b205e4, 843512b2aa644b55a21b7e702943e02f
I20260709 12:17:21.261530 22107 tablet_replica.cc:333] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.261842 32671 raft_consensus.cc:2749] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.262298 22107 raft_consensus.cc:2243] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.262804 22107 raft_consensus.cc:2272] T 544954717af441188c2a2b0ff898f279 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.264707 22107 tablet_replica.cc:333] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.265305 22107 raft_consensus.cc:2243] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.265792 22107 raft_consensus.cc:2272] T e6c8433c534b4d19bc10a5cb0ca094ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.268535 22107 tablet_replica.cc:333] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.269402 22107 raft_consensus.cc:2243] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.269992 22107 raft_consensus.cc:2272] T fe911dcdb0624a2293e42d3d26312b21 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.272161 22107 tablet_replica.cc:333] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.272847 22107 raft_consensus.cc:2243] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.273794 22107 raft_consensus.cc:2272] T 78d7a21102b64e7f95625a42c84e9ad0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.275796 32671 raft_consensus.cc:493] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 7dd13c0178c84f7aaab5c435a8b205e4)
I20260709 12:17:21.275954 22107 tablet_replica.cc:333] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.276314 32671 raft_consensus.cc:515] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd0f6deb3b147509a008c574d9d879d" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 38461 } } peers { permanent_uuid: "bf0f149240cc4f27b97810c90eba7ff9" member_type: VOTER last_known_addr { host: "127.21.150.196" port: 40451 } } peers { permanent_uuid: "7dd13c0178c84f7aaab5c435a8b205e4" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 37911 } }
I20260709 12:17:21.276731 22107 raft_consensus.cc:2243] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.277653 22107 raft_consensus.cc:2272] T e302635e586c48819cf549549c5f4446 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.278306 32671 leader_election.cc:290] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461), 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911)
I20260709 12:17:21.280139 22107 tablet_replica.cc:333] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.280943 22107 raft_consensus.cc:2243] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.281473 22107 raft_consensus.cc:2272] T 384f88a7446842bfb4a6c71e921c9049 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:21.282872 31693 leader_election.cc:336] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bd0f6deb3b147509a008c574d9d879d (127.21.150.195:38461): Network error: Client connection negotiation failed: client connection to 127.21.150.195:38461: connect: Connection refused (error 111)
I20260709 12:17:21.284145 22107 tablet_replica.cc:333] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
W20260709 12:17:21.284091 31690 leader_election.cc:336] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7dd13c0178c84f7aaab5c435a8b205e4 (127.21.150.193:37911): Network error: Client connection negotiation failed: client connection to 127.21.150.193:37911: connect: Connection refused (error 111)
I20260709 12:17:21.284662 31690 leader_election.cc:304] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf0f149240cc4f27b97810c90eba7ff9; no voters: 0bd0f6deb3b147509a008c574d9d879d, 7dd13c0178c84f7aaab5c435a8b205e4
I20260709 12:17:21.284927 22107 raft_consensus.cc:2243] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.285305 32671 raft_consensus.cc:2749] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:21.285512 22107 raft_consensus.cc:2272] T a2cd62a9ae5d4430bdb3799b63ca2137 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.287712 22107 tablet_replica.cc:333] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.288319 22107 raft_consensus.cc:2243] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.288738 22107 raft_consensus.cc:2272] T 71ec83ad93ff4b3abcf4fe9f6005aa7c P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.290647 22107 tablet_replica.cc:333] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.291191 22107 raft_consensus.cc:2243] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.291625 22107 raft_consensus.cc:2272] T 43ac1bb8ec79464ebaa8b8d0d4a5747d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.293504 22107 tablet_replica.cc:333] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.294216 22107 raft_consensus.cc:2243] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.294703 22107 raft_consensus.cc:2272] T 5590853f22ef45de955f77e7648a574b P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.297129 22107 tablet_replica.cc:333] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.297886 22107 raft_consensus.cc:2243] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.298372 22107 raft_consensus.cc:2272] T 3c241197f69848f4b2142121633ec9ef P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.300701 22107 tablet_replica.cc:333] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.301455 22107 raft_consensus.cc:2243] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.301975 22107 raft_consensus.cc:2272] T ecff280f5652462db2e6473ea7424a2f P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.305022 22107 tablet_replica.cc:333] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.305795 22107 raft_consensus.cc:2243] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.306779 22107 raft_consensus.cc:2272] T a6c957e6781843b9b69e2d98053daa74 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.309650 22107 tablet_replica.cc:333] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.310388 22107 raft_consensus.cc:2243] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.311324 22107 raft_consensus.cc:2272] T 6d00a005dd8945d9acac25d9822223a6 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.313920 22107 tablet_replica.cc:333] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.314641 22107 raft_consensus.cc:2243] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.315119 22107 raft_consensus.cc:2272] T dbfef93226df40f3a8d07af88af622fb P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.317476 22107 tablet_replica.cc:333] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.318226 22107 raft_consensus.cc:2243] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.318843 22107 raft_consensus.cc:2272] T 59dc1ab82c334d8f8954116a598b7c8b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.321461 22107 tablet_replica.cc:333] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.322269 22107 raft_consensus.cc:2243] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.322808 22107 raft_consensus.cc:2272] T 81a98a7e3a4c46e9a01e39bb9924afc0 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.325456 22107 tablet_replica.cc:333] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.326216 22107 raft_consensus.cc:2243] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.326743 22107 raft_consensus.cc:2272] T a44cca173ddc49d99e53c40bccf8b5c9 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.329221 22107 tablet_replica.cc:333] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.329948 22107 raft_consensus.cc:2243] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.330416 22107 raft_consensus.cc:2272] T a194f9fff2f94f8d8ecfe700ee83d998 P bf0f149240cc4f27b97810c90eba7ff9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.332795 22107 tablet_replica.cc:333] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.333508 22107 raft_consensus.cc:2243] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.334013 22107 raft_consensus.cc:2272] T b667dda9be9d46cdba95860c4c37102e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.336621 22107 tablet_replica.cc:333] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.337322 22107 raft_consensus.cc:2243] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.337893 22107 raft_consensus.cc:2272] T 4dfd13293d3149f6a1ca73fd10852083 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.340476 22107 tablet_replica.cc:333] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.341214 22107 raft_consensus.cc:2243] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.342254 22107 raft_consensus.cc:2272] T 1f6baf18d4c74681ab192824db201eec P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.344866 22107 tablet_replica.cc:333] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.345621 22107 raft_consensus.cc:2243] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.346143 22107 raft_consensus.cc:2272] T dc169af544e14e668b85ec4da6ca648d P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.348721 22107 tablet_replica.cc:333] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.349515 22107 raft_consensus.cc:2243] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.349987 22107 raft_consensus.cc:2272] T 53b9a430b2d54764b10115eae2e0e653 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.352385 22107 tablet_replica.cc:333] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.353114 22107 raft_consensus.cc:2243] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.354144 22107 raft_consensus.cc:2272] T 74bd4922c2764afd8737751cb3d17cb8 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.356676 22107 tablet_replica.cc:333] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.357522 22107 raft_consensus.cc:2243] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.358032 22107 raft_consensus.cc:2272] T 4c6ef1dbe6184d41b2a5edfb45c8315e P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.360450 22107 tablet_replica.cc:333] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.361145 22107 raft_consensus.cc:2243] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.362090 22107 raft_consensus.cc:2272] T b76fc75555e64cd3a5762910676fb4e3 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.364713 22107 tablet_replica.cc:333] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.365478 22107 raft_consensus.cc:2243] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.366463 22107 raft_consensus.cc:2272] T 506b4da2b0be46d5bf3ff52167d4419b P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.369166 22107 tablet_replica.cc:333] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.369889 22107 raft_consensus.cc:2243] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.370429 22107 raft_consensus.cc:2272] T 75b15073c4e54ef1af602f9c83702642 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.372685 22107 tablet_replica.cc:333] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.373247 22107 raft_consensus.cc:2243] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.374158 22107 raft_consensus.cc:2272] T 098e97adee224f86ba1bfa07ce392b68 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.376856 22107 tablet_replica.cc:333] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.377566 22107 raft_consensus.cc:2243] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.378574 22107 raft_consensus.cc:2272] T af22442d6a9141f2a4607ba57cf4e0ba P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.380517 22107 tablet_replica.cc:333] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.381064 22107 raft_consensus.cc:2243] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.381546 22107 raft_consensus.cc:2272] T 6019e17e78a14e9fa70a4f5aa6e26056 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.383409 22107 tablet_replica.cc:333] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.383932 22107 raft_consensus.cc:2243] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.384655 22107 raft_consensus.cc:2272] T 45a8924a0b9c4dbc9ace49508d7b9f7e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.386984 22107 tablet_replica.cc:333] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.387512 22107 raft_consensus.cc:2243] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.387934 22107 raft_consensus.cc:2272] T 9ee706eb478f44e0979857553c460e94 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.389858 22107 tablet_replica.cc:333] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.390376 22107 raft_consensus.cc:2243] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.390805 22107 raft_consensus.cc:2272] T 096e1ab4f2704ddaaad535361f46d6a2 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.392553 22107 tablet_replica.cc:333] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.393078 22107 raft_consensus.cc:2243] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.393959 22107 raft_consensus.cc:2272] T 28ff8e905a7b4eb5b2cea1df313efa14 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.396193 22107 tablet_replica.cc:333] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.397003 22107 raft_consensus.cc:2243] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.397578 22107 raft_consensus.cc:2272] T 3f18f4e3f28a4fa7844c90b02bea7476 P bf0f149240cc4f27b97810c90eba7ff9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.400192 22107 tablet_replica.cc:333] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.400807 22107 raft_consensus.cc:2243] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.401865 22107 raft_consensus.cc:2272] T dc5b5e283ec34f03b184be6dfa4e046e P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.407734 22107 tablet_replica.cc:333] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.408298 22107 raft_consensus.cc:2243] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.408737 22107 raft_consensus.cc:2272] T 8e0f4df696784a248aaa0b98db69dfa0 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.410723 22107 tablet_replica.cc:333] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.411226 22107 raft_consensus.cc:2243] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.411643 22107 raft_consensus.cc:2272] T 0281541aef814bce9263539d80488282 P bf0f149240cc4f27b97810c90eba7ff9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.413393 22107 tablet_replica.cc:333] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.413921 22107 raft_consensus.cc:2243] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.414705 22107 raft_consensus.cc:2272] T fd72db62f85b4234aee25b5eda5abddd P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.416821 22107 tablet_replica.cc:333] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9: stopping tablet replica
I20260709 12:17:21.417291 22107 raft_consensus.cc:2243] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:21.417764 22107 raft_consensus.cc:2272] T b52b6a2957cc45ae96342d2b42cf8ae1 P bf0f149240cc4f27b97810c90eba7ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.485261 22107 tablet_server.cc:196] TabletServer@127.21.150.196:0 shutdown complete.
I20260709 12:17:21.549943 22107 master.cc:562] Master@127.21.150.254:39763 shutting down...
I20260709 12:17:21.576014 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:21.576795 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:21.577171 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1f0c8e00074048c1901b985892084ace: stopping tablet replica
I20260709 12:17:21.599916 22107 master.cc:584] Master@127.21.150.254:39763 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (31573 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:17:21.686635 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:43431
I20260709 12:17:21.687826 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:21.694024 32674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:21.694123 32673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:21.694607 32676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:21.697364 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:21.698454 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:21.698678 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:21.698845 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599441698828 us; error 0 us; skew 500 ppm
I20260709 12:17:21.699388 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:21.701969 22107 webserver.cc:533] Webserver started at http://127.21.150.254:40601/ using document root <none> and password file <none>
I20260709 12:17:21.702467 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:21.702673 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:21.702942 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:21.704070 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "8bcd7c280ec14efa90134a5a1346a12b"
format_stamp: "Formatted at 2026-07-09 12:17:21 on dist-test-slave-zctp"
I20260709 12:17:21.708514 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:21.712350 32681 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:21.713265 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:17:21.713574 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "8bcd7c280ec14efa90134a5a1346a12b"
format_stamp: "Formatted at 2026-07-09 12:17:21 on dist-test-slave-zctp"
I20260709 12:17:21.713826 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:21.728503 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:21.729748 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:21.769490 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:43431
I20260709 12:17:21.769621 32732 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:43431 every 8 connection(s)
I20260709 12:17:21.773461 32733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:21.784627 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b: Bootstrap starting.
I20260709 12:17:21.789546 32733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:21.793792 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b: No bootstrap required, opened a new log
I20260709 12:17:21.795954 32733 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER }
I20260709 12:17:21.796361 32733 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:21.796583 32733 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bcd7c280ec14efa90134a5a1346a12b, State: Initialized, Role: FOLLOWER
I20260709 12:17:21.797293 32733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER }
I20260709 12:17:21.797782 32733 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:21.798007 32733 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:21.798286 32733 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:21.802995 32733 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER }
I20260709 12:17:21.803524 32733 leader_election.cc:304] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bcd7c280ec14efa90134a5a1346a12b; no voters: 
I20260709 12:17:21.804653 32733 leader_election.cc:290] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:21.805035 32736 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:21.806464 32736 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 1 LEADER]: Becoming Leader. State: Replica: 8bcd7c280ec14efa90134a5a1346a12b, State: Running, Role: LEADER
I20260709 12:17:21.807163 32736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER }
I20260709 12:17:21.807703 32733 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:21.811415 32738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bcd7c280ec14efa90134a5a1346a12b. Latest consensus state: current_term: 1 leader_uuid: "8bcd7c280ec14efa90134a5a1346a12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER } }
I20260709 12:17:21.812197 32738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:21.811450 32737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bcd7c280ec14efa90134a5a1346a12b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bcd7c280ec14efa90134a5a1346a12b" member_type: VOTER } }
I20260709 12:17:21.813342 32737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:21.814924 32742 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:21.823736 32742 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:21.824860 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:21.832860 32742 catalog_manager.cc:1383] Generated new cluster ID: a55592588d2942eaac2e1301df2fdefa
I20260709 12:17:21.833196 32742 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:21.846841 32742 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:21.848205 32742 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:21.864464 32742 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b: Generated new TSK 0
I20260709 12:17:21.865116 32742 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:21.892319 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:21.898615 32754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:21.899777 32755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:21.901422 22107 server_base.cc:1061] running on GCE node
W20260709 12:17:21.901337 32757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:21.902685 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:21.902884 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:21.903007 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599441902996 us; error 0 us; skew 500 ppm
I20260709 12:17:21.903478 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:21.905941 22107 webserver.cc:533] Webserver started at http://127.21.150.193:33769/ using document root <none> and password file <none>
I20260709 12:17:21.906380 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:21.906539 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:21.906750 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:21.907770 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "efe9c93a145a490daba616e633c14a8b"
format_stamp: "Formatted at 2026-07-09 12:17:21 on dist-test-slave-zctp"
I20260709 12:17:21.912410 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:17:21.916252 32762 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:21.917029 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:17:21.917377 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "efe9c93a145a490daba616e633c14a8b"
format_stamp: "Formatted at 2026-07-09 12:17:21 on dist-test-slave-zctp"
I20260709 12:17:21.917773 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:21.938526 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:21.939719 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:21.941167 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:21.943481 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:21.943722 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:21.943964 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:21.944157 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:21.986225 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:44501
I20260709 12:17:21.986326   356 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:44501 every 8 connection(s)
I20260709 12:17:21.991359 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:22.000202   361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:22.000849   362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:22.005532 22107 server_base.cc:1061] running on GCE node
W20260709 12:17:22.006421   364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:22.007951 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:22.008239 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:22.008438 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599442008418 us; error 0 us; skew 500 ppm
I20260709 12:17:22.009107 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:22.010596   357 heartbeater.cc:344] Connected to a master server at 127.21.150.254:43431
I20260709 12:17:22.011018   357 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:22.011883 22107 webserver.cc:533] Webserver started at http://127.21.150.194:41631/ using document root <none> and password file <none>
I20260709 12:17:22.011917   357 heartbeater.cc:507] Master 127.21.150.254:43431 requested a full tablet report, sending...
I20260709 12:17:22.012478 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:22.012686 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:22.012894 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:22.014772 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "7dde1735325f414bbdb50b608d80dca1"
format_stamp: "Formatted at 2026-07-09 12:17:22 on dist-test-slave-zctp"
I20260709 12:17:22.015161 32698 ts_manager.cc:194] Registered new tserver with Master: efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501)
I20260709 12:17:22.017880 32698 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40660
I20260709 12:17:22.021339 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:17:22.025230   369 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.026240 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:22.026549 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "7dde1735325f414bbdb50b608d80dca1"
format_stamp: "Formatted at 2026-07-09 12:17:22 on dist-test-slave-zctp"
I20260709 12:17:22.026815 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:22.046762 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:22.047979 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:22.049394 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:22.051692 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:22.051877 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.052155 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:22.052361 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.099468 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:40963
I20260709 12:17:22.099658   431 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:40963 every 8 connection(s)
I20260709 12:17:22.104485 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:22.113379   436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:22.114629   435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:22.117775   432 heartbeater.cc:344] Connected to a master server at 127.21.150.254:43431
W20260709 12:17:22.118060   438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:22.118346   432 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:22.119412   432 heartbeater.cc:507] Master 127.21.150.254:43431 requested a full tablet report, sending...
I20260709 12:17:22.120513 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:22.121750 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:22.122001 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:22.122243 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599442122207 us; error 0 us; skew 500 ppm
I20260709 12:17:22.122582 32698 ts_manager.cc:194] Registered new tserver with Master: 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.122807 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:22.124454 32698 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40674
I20260709 12:17:22.125849 22107 webserver.cc:533] Webserver started at http://127.21.150.195:38761/ using document root <none> and password file <none>
I20260709 12:17:22.126459 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:22.126689 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:22.126979 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:22.128175 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "784b9febfff245dda6a27c4be73884fe"
format_stamp: "Formatted at 2026-07-09 12:17:22 on dist-test-slave-zctp"
I20260709 12:17:22.132786 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:17:22.136699   443 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.137558 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:22.137861 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "784b9febfff245dda6a27c4be73884fe"
format_stamp: "Formatted at 2026-07-09 12:17:22 on dist-test-slave-zctp"
I20260709 12:17:22.138178 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:22.155295 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:22.156554 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:22.158186 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:22.160465 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:22.160702 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.160950 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:22.161134 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:22.201382 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:43377
I20260709 12:17:22.201541   505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:43377 every 8 connection(s)
I20260709 12:17:22.215248   506 heartbeater.cc:344] Connected to a master server at 127.21.150.254:43431
I20260709 12:17:22.215600   506 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:22.216374   506 heartbeater.cc:507] Master 127.21.150.254:43431 requested a full tablet report, sending...
I20260709 12:17:22.218815 32698 ts_manager.cc:194] Registered new tserver with Master: 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:22.219640 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014861879s
I20260709 12:17:22.220590 32698 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40686
I20260709 12:17:22.244834 32698 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40700:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:22.247803 32698 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:22.337584   322 tablet_service.cc:1511] Processing CreateTablet for tablet a761e98905764ff5af3aa30229ca50d4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:22.338447   396 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3b5891cbb48d19e8dead71573964b (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:22.339110   322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a761e98905764ff5af3aa30229ca50d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.339919   396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3b5891cbb48d19e8dead71573964b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.340342   471 tablet_service.cc:1511] Processing CreateTablet for tablet a761e98905764ff5af3aa30229ca50d4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:22.340631   397 tablet_service.cc:1511] Processing CreateTablet for tablet a761e98905764ff5af3aa30229ca50d4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:17:22.341739   471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a761e98905764ff5af3aa30229ca50d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.341967   397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a761e98905764ff5af3aa30229ca50d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.344496   470 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3b5891cbb48d19e8dead71573964b (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:22.345883   470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3b5891cbb48d19e8dead71573964b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.346596   469 tablet_service.cc:1511] Processing CreateTablet for tablet 11ee1edff6824b32bb78ced08ffba7f5 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:22.347110   395 tablet_service.cc:1511] Processing CreateTablet for tablet 11ee1edff6824b32bb78ced08ffba7f5 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:22.347967   469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ee1edff6824b32bb78ced08ffba7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.348428   395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ee1edff6824b32bb78ced08ffba7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.350525   468 tablet_service.cc:1511] Processing CreateTablet for tablet aaae3f0f5bc646a6aeb204604f7d99d3 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:22.351867   468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaae3f0f5bc646a6aeb204604f7d99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.351719   465 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4d76718f446c78e1feb938aefc903 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:22.353062   465 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4d76718f446c78e1feb938aefc903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.354606   467 tablet_service.cc:1511] Processing CreateTablet for tablet 75e138f0842544c9903c3e030689ccb7 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:22.355014   394 tablet_service.cc:1511] Processing CreateTablet for tablet aaae3f0f5bc646a6aeb204604f7d99d3 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:22.355973   467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e138f0842544c9903c3e030689ccb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.356279   394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaae3f0f5bc646a6aeb204604f7d99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.359637   393 tablet_service.cc:1511] Processing CreateTablet for tablet 75e138f0842544c9903c3e030689ccb7 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:22.360978   393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e138f0842544c9903c3e030689ccb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.361123   464 tablet_service.cc:1511] Processing CreateTablet for tablet f2a3bb67994c4a9e804975daa26f99f6 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:22.350539   466 tablet_service.cc:1511] Processing CreateTablet for tablet 56b6194d92c34c089bf0c6a8ecef5930 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:22.362466   464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2a3bb67994c4a9e804975daa26f99f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.363801   463 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0d9e0f0f949099570bddfa5106e8f (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:22.365113   463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0d9e0f0f949099570bddfa5106e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.366353   466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b6194d92c34c089bf0c6a8ecef5930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.370525   321 tablet_service.cc:1511] Processing CreateTablet for tablet 49b3b5891cbb48d19e8dead71573964b (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:17:22.371824   321 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49b3b5891cbb48d19e8dead71573964b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.379232   320 tablet_service.cc:1511] Processing CreateTablet for tablet 11ee1edff6824b32bb78ced08ffba7f5 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:17:22.380712   320 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ee1edff6824b32bb78ced08ffba7f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.383884   462 tablet_service.cc:1511] Processing CreateTablet for tablet d249715915f64545ba53f74e84a294e4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:22.385324   319 tablet_service.cc:1511] Processing CreateTablet for tablet aaae3f0f5bc646a6aeb204604f7d99d3 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:17:22.385356   462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d249715915f64545ba53f74e84a294e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.386766   319 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaae3f0f5bc646a6aeb204604f7d99d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.395603   392 tablet_service.cc:1511] Processing CreateTablet for tablet 56b6194d92c34c089bf0c6a8ecef5930 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:22.397298   392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b6194d92c34c089bf0c6a8ecef5930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.398013   390 tablet_service.cc:1511] Processing CreateTablet for tablet f2a3bb67994c4a9e804975daa26f99f6 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:22.399395   390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2a3bb67994c4a9e804975daa26f99f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.395774   391 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4d76718f446c78e1feb938aefc903 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:22.402307   391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4d76718f446c78e1feb938aefc903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.406694   389 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0d9e0f0f949099570bddfa5106e8f (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:22.408087   389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0d9e0f0f949099570bddfa5106e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.414999   318 tablet_service.cc:1511] Processing CreateTablet for tablet 75e138f0842544c9903c3e030689ccb7 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:17:22.417694   318 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75e138f0842544c9903c3e030689ccb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.417538   317 tablet_service.cc:1511] Processing CreateTablet for tablet 56b6194d92c34c089bf0c6a8ecef5930 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:17:22.421176   314 tablet_service.cc:1511] Processing CreateTablet for tablet 58d0d9e0f0f949099570bddfa5106e8f (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:17:22.424180   315 tablet_service.cc:1511] Processing CreateTablet for tablet f2a3bb67994c4a9e804975daa26f99f6 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:17:22.426757   316 tablet_service.cc:1511] Processing CreateTablet for tablet 84f4d76718f446c78e1feb938aefc903 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:17:22.418954   313 tablet_service.cc:1511] Processing CreateTablet for tablet d249715915f64545ba53f74e84a294e4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:22.445904   317 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b6194d92c34c089bf0c6a8ecef5930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.448576   316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84f4d76718f446c78e1feb938aefc903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.450112   314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58d0d9e0f0f949099570bddfa5106e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.452374   313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d249715915f64545ba53f74e84a294e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.453971   315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2a3bb67994c4a9e804975daa26f99f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.478855   521 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.511935   523 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:22.522346   388 tablet_service.cc:1511] Processing CreateTablet for tablet d249715915f64545ba53f74e84a294e4 (DEFAULT_TABLE table=test-workload [id=4c8f8669872649718618aedc23c0977a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:17:22.530002   522 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.533247   388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d249715915f64545ba53f74e84a294e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:22.556612   522 tablet_bootstrap.cc:654] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.590122   521 tablet_bootstrap.cc:654] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.592343   523 tablet_bootstrap.cc:654] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.598486   522 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:22.602278   522 ts_tablet_manager.cc:1403] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.073s	user 0.007s	sys 0.007s
I20260709 12:17:22.606267   522 raft_consensus.cc:359] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.607568   522 raft_consensus.cc:385] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.608107   522 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.607147   521 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:22.609362   521 ts_tablet_manager.cc:1403] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.014s
I20260709 12:17:22.610081   523 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:22.610694   523 ts_tablet_manager.cc:1403] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.099s	user 0.019s	sys 0.000s
I20260709 12:17:22.613246   522 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.613871   523 raft_consensus.cc:359] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.615635   523 raft_consensus.cc:385] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.615156   521 raft_consensus.cc:359] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.615974   523 raft_consensus.cc:740] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.616132   521 raft_consensus.cc:385] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.616958   521 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.617040   523 consensus_queue.cc:260] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.618377   521 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.619689   522 ts_tablet_manager.cc:1434] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:17:22.620828   432 heartbeater.cc:499] Master 127.21.150.254:43431 was elected leader, sending a full tablet report...
I20260709 12:17:22.621074   522 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.627521   357 heartbeater.cc:499] Master 127.21.150.254:43431 was elected leader, sending a full tablet report...
I20260709 12:17:22.627846   523 ts_tablet_manager.cc:1434] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.008s
I20260709 12:17:22.629025   523 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:22.630007   522 tablet_bootstrap.cc:654] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.635548   523 tablet_bootstrap.cc:654] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.647601   523 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:22.648216   523 ts_tablet_manager.cc:1403] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:17:22.651420   523 raft_consensus.cc:359] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.652422   523 raft_consensus.cc:385] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.652791   523 raft_consensus.cc:740] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.653803   523 consensus_queue.cc:260] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.656388   523 ts_tablet_manager.cc:1434] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:17:22.657474   523 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:22.664137   523 tablet_bootstrap.cc:654] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.690927   506 heartbeater.cc:499] Master 127.21.150.254:43431 was elected leader, sending a full tablet report...
I20260709 12:17:22.691903   521 ts_tablet_manager.cc:1434] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.081s	user 0.026s	sys 0.049s
I20260709 12:17:22.693382   521 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.702342   521 tablet_bootstrap.cc:654] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:22.705799   507 tablet.cc:2406] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:17:22.712793   522 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:22.713178   523 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:22.713184   521 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:22.713395   522 ts_tablet_manager.cc:1403] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.093s	user 0.041s	sys 0.049s
I20260709 12:17:22.713932   521 ts_tablet_manager.cc:1403] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:17:22.717821   522 raft_consensus.cc:359] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.719659   522 raft_consensus.cc:385] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.719991   522 raft_consensus.cc:740] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.713929   523 ts_tablet_manager.cc:1403] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.057s	user 0.027s	sys 0.028s
I20260709 12:17:22.720943   522 consensus_queue.cc:260] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.722101   521 raft_consensus.cc:359] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.722900   521 raft_consensus.cc:385] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.723286   522 ts_tablet_manager.cc:1434] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:17:22.723239   521 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.724114   523 raft_consensus.cc:359] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.724679   522 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.724388   521 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.724889   523 raft_consensus.cc:385] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.725267   523 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.726102   523 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.727170   521 ts_tablet_manager.cc:1434] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:17:22.730298   521 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.732174   522 tablet_bootstrap.cc:654] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.728754   523 ts_tablet_manager.cc:1434] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:17:22.737661   522 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:22.738387   522 ts_tablet_manager.cc:1403] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20260709 12:17:22.738905   521 tablet_bootstrap.cc:654] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.740015   523 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:22.743501   522 raft_consensus.cc:359] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.744385   522 raft_consensus.cc:385] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.744736   522 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.745790   522 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.747920   523 tablet_bootstrap.cc:654] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.748593   522 ts_tablet_manager.cc:1434] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:17:22.749893   522 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.750432   521 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:22.751019   521 ts_tablet_manager.cc:1403] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.012s
I20260709 12:17:22.754043   521 raft_consensus.cc:359] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.754930   521 raft_consensus.cc:385] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.755275   521 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.756227   521 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.758553   522 tablet_bootstrap.cc:654] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.761333   521 ts_tablet_manager.cc:1434] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:17:22.762480   521 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.769093   521 tablet_bootstrap.cc:654] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.772238   523 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:22.772909   523 ts_tablet_manager.cc:1403] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.016s
I20260709 12:17:22.773465   522 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:22.774094   522 ts_tablet_manager.cc:1403] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.014s
I20260709 12:17:22.776026   523 raft_consensus.cc:359] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:22.776686   523 raft_consensus.cc:385] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.776937   523 raft_consensus.cc:740] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.777372   521 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:22.777081   522 raft_consensus.cc:359] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.777967   521 ts_tablet_manager.cc:1403] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 12:17:22.777923   522 raft_consensus.cc:385] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.777702   523 consensus_queue.cc:260] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:22.778517   522 raft_consensus.cc:740] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.779598   522 consensus_queue.cc:260] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.780224   523 ts_tablet_manager.cc:1434] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:17:22.781327   523 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:22.781258   521 raft_consensus.cc:359] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.781879   529 raft_consensus.cc:493] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.782200   521 raft_consensus.cc:385] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.782593   521 raft_consensus.cc:740] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.782437   529 raft_consensus.cc:515] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.783627   521 consensus_queue.cc:260] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.789263   523 tablet_bootstrap.cc:654] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.790771   522 ts_tablet_manager.cc:1434] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:17:22.796329   522 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.805409   522 tablet_bootstrap.cc:654] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.807753   521 ts_tablet_manager.cc:1434] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.029s	user 0.017s	sys 0.011s
I20260709 12:17:22.811586   521 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.813572   529 leader_election.cc:290] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.816473   521 tablet_bootstrap.cc:654] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:22.818858 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:22.848278   528 raft_consensus.cc:493] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.848899   528 raft_consensus.cc:515] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:22.858165   528 leader_election.cc:290] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.860759   528 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.862394   528 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.866240   528 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:22.868731   522 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:22.869400   522 ts_tablet_manager.cc:1403] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.073s	user 0.019s	sys 0.036s
I20260709 12:17:22.871451   521 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:22.872004   521 ts_tablet_manager.cc:1403] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.061s	user 0.027s	sys 0.027s
I20260709 12:17:22.872762   522 raft_consensus.cc:359] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.873626   522 raft_consensus.cc:385] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.873968   522 raft_consensus.cc:740] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.874917   522 consensus_queue.cc:260] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.877188   522 ts_tablet_manager.cc:1434] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:17:22.875053   521 raft_consensus.cc:359] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.877714   521 raft_consensus.cc:385] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.878104   521 raft_consensus.cc:740] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.879087   521 consensus_queue.cc:260] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.881446   521 ts_tablet_manager.cc:1434] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:17:22.879491   522 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:22.887288   527 raft_consensus.cc:493] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.888280   521 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:22.887960   527 raft_consensus.cc:515] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.892460   527 leader_election.cc:290] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:22.894780   522 tablet_bootstrap.cc:654] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.894842   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
I20260709 12:17:22.899541   521 tablet_bootstrap.cc:654] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:22.904474   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:22.905118   529 raft_consensus.cc:493] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.905292   407 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 0.
I20260709 12:17:22.905642   529 raft_consensus.cc:515] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.907826   529 leader_election.cc:290] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.907994   445 leader_election.cc:304] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:22.910207   529 raft_consensus.cc:2804] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:22.910688   529 raft_consensus.cc:493] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.911099   529 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.917673   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
W20260709 12:17:22.918040   447 leader_election.cc:343] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:22.918447   407 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 0.
I20260709 12:17:22.917677   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
W20260709 12:17:22.922246   447 leader_election.cc:343] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:22.924680   445 leader_election.cc:304] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:22.930349   539 raft_consensus.cc:2804] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:22.930752   539 raft_consensus.cc:493] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.931077   539 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.933502   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:22.935195   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
W20260709 12:17:22.936830 32764 leader_election.cc:343] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:22.937546 32764 leader_election.cc:343] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:22.939141   539 raft_consensus.cc:515] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.946199   539 leader_election.cc:290] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.947672   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:22.953966   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
I20260709 12:17:22.954619   332 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 0.
W20260709 12:17:22.955925   447 leader_election.cc:343] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:22.948792   523 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:22.948678   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:22.951705   528 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:22.956879   407 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.957018   528 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.956729   373 leader_election.cc:304] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:22.959997   528 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:22.960144   527 raft_consensus.cc:2804] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:22.960608   527 raft_consensus.cc:493] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.960958   527 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.962538   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:22.963454   406 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:22.964977   407 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:22.964949 32764 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:22.966426   528 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:22.966630   445 leader_election.cc:304] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:22.967013   528 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:22.967588   528 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.968266   523 ts_tablet_manager.cc:1403] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.187s	user 0.033s	sys 0.036s
I20260709 12:17:22.968559   527 raft_consensus.cc:515] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.971566   523 raft_consensus.cc:359] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.972553   539 raft_consensus.cc:2804] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:22.978860   539 raft_consensus.cc:697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:22.973563   529 raft_consensus.cc:515] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.977972   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:22.979795   527 leader_election.cc:290] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:22.980172   332 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.980538   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:22.979795   539 consensus_queue.cc:237] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:22.981712   481 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 0.
I20260709 12:17:22.984613   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:22.985306   481 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.987774   529 leader_election.cc:290] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:22.989764   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:22.990392   407 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:22.990917   331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:22.991560   332 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
W20260709 12:17:22.992467   447 leader_election.cc:343] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:22.992799   373 leader_election.cc:304] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:22.996531   480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:22.997328   523 raft_consensus.cc:385] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:22.997738   523 raft_consensus.cc:740] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:22.998075   479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:22.998651   523 consensus_queue.cc:260] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.000047   478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:22.998975   479 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.001549   523 ts_tablet_manager.cc:1434] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:17:23.002663   523 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
W20260709 12:17:23.002732 32764 leader_election.cc:343] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.003810 32764 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 7dde1735325f414bbdb50b608d80dca1
W20260709 12:17:23.005007 32764 leader_election.cc:343] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.006410   481 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
I20260709 12:17:23.009933   528 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.011061 32764 leader_election.cc:304] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe9c93a145a490daba616e633c14a8b; no voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:23.006320   545 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.012267   545 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.012697   545 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.013556   528 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Requested vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.019905   521 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:23.020432   522 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:23.020568   523 tablet_bootstrap.cc:654] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.020572   521 ts_tablet_manager.cc:1403] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.133s	user 0.040s	sys 0.070s
I20260709 12:17:23.020949   522 ts_tablet_manager.cc:1403] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.142s	user 0.034s	sys 0.037s
I20260709 12:17:22.996641   527 raft_consensus.cc:2804] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.022492   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
W20260709 12:17:23.026096 32764 leader_election.cc:343] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.026671   522 raft_consensus.cc:359] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.027536   522 raft_consensus.cc:385] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.027311   521 raft_consensus.cc:359] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.028038   522 raft_consensus.cc:740] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.028131   521 raft_consensus.cc:385] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.028644   521 raft_consensus.cc:740] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.029194   522 consensus_queue.cc:260] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.029604   521 consensus_queue.cc:260] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.031606   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.031740   528 raft_consensus.cc:2749] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:23.032032   521 ts_tablet_manager.cc:1434] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:17:23.032224   406 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.033053   523 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:23.033205   521 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:23.033613   523 ts_tablet_manager.cc:1403] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.012s
I20260709 12:17:23.034747   545 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.036700   523 raft_consensus.cc:359] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.037508   523 raft_consensus.cc:385] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.037879   407 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:23.037981   523 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.039227   445 leader_election.cc:304] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:23.040109   521 tablet_bootstrap.cc:654] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.040186   555 raft_consensus.cc:2804] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.041078   555 raft_consensus.cc:697] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:23.041808   405 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.040424   406 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.042184   555 consensus_queue.cc:237] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
W20260709 12:17:23.043629 32764 leader_election.cc:343] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.044484 32764 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:23.045982   528 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.039069   523 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.048760   545 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Requested vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.049510   527 raft_consensus.cc:697] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:23.049927   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:23.050683   481 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.050715   527 consensus_queue.cc:237] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.060632   481 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.061666   521 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:23.062240   521 ts_tablet_manager.cc:1403] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.005s
I20260709 12:17:23.062108 32764 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:23.050169   522 ts_tablet_manager.cc:1434] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.004s
I20260709 12:17:23.064286   522 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:23.065354   521 raft_consensus.cc:359] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.066172   521 raft_consensus.cc:385] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.066594   521 raft_consensus.cc:740] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.070508 32694 catalog_manager.cc:5697] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.074986   523 ts_tablet_manager.cc:1434] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.041s	user 0.011s	sys 0.006s
I20260709 12:17:23.075059   522 tablet_bootstrap.cc:654] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.074440 32695 catalog_manager.cc:5697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 1 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:23.076236   523 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:23.077554   545 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.078579   545 raft_consensus.cc:697] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Becoming Leader. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Running, Role: LEADER
I20260709 12:17:23.079378   528 raft_consensus.cc:697] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Becoming Leader. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Running, Role: LEADER
I20260709 12:17:23.079214   521 consensus_queue.cc:260] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.081370   528 consensus_queue.cc:237] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.081343   545 consensus_queue.cc:237] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.085629   521 ts_tablet_manager.cc:1434] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 12:17:23.086939   521 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:23.091634   523 tablet_bootstrap.cc:654] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.095067   521 tablet_bootstrap.cc:654] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.104923 32696 catalog_manager.cc:5697] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to efe9c93a145a490daba616e633c14a8b (127.21.150.193). New cstate: current_term: 1 leader_uuid: "efe9c93a145a490daba616e633c14a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.111200   522 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:23.112180   522 ts_tablet_manager.cc:1403] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.018s
I20260709 12:17:23.116477   522 raft_consensus.cc:359] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.117662   522 raft_consensus.cc:385] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.118147   522 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.119616   522 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.123298   522 ts_tablet_manager.cc:1434] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:17:23.124343   522 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:23.125113   539 raft_consensus.cc:493] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.125710   539 raft_consensus.cc:515] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.129680   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
I20260709 12:17:23.131199   522 tablet_bootstrap.cc:654] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.132511   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
W20260709 12:17:23.131223   447 leader_election.cc:343] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.133204   407 raft_consensus.cc:2468] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 0.
I20260709 12:17:23.134617   527 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.135070   521 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:23.135432   445 leader_election.cc:304] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:23.135139   527 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.135947   521 ts_tablet_manager.cc:1403] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.049s	user 0.036s	sys 0.005s
I20260709 12:17:23.136435   555 raft_consensus.cc:2804] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.136826   555 raft_consensus.cc:493] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.137141   555 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.138501   527 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.139552   521 raft_consensus.cc:359] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.140511   521 raft_consensus.cc:385] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.140851   521 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.141871   521 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.143260   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
I20260709 12:17:23.144948   521 ts_tablet_manager.cc:1434] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:17:23.146271   521 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:23.146641   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:23.147795   481 raft_consensus.cc:2393] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7dde1735325f414bbdb50b608d80dca1 in current term 1: Already voted for candidate efe9c93a145a490daba616e633c14a8b in this term.
I20260709 12:17:23.149824   371 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1; no voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b
I20260709 12:17:23.150915   527 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.153579   539 leader_election.cc:290] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:23.154404   521 tablet_bootstrap.cc:654] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.158349   527 raft_consensus.cc:2749] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:23.158643   523 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:23.159320   523 ts_tablet_manager.cc:1403] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.019s
I20260709 12:17:23.163779   523 raft_consensus.cc:359] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.156931   555 raft_consensus.cc:515] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.167761   521 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:23.170764   521 ts_tablet_manager.cc:1403] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.001s
I20260709 12:17:23.173132   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:23.173689   521 raft_consensus.cc:359] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.174446   521 raft_consensus.cc:385] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.174708   521 raft_consensus.cc:740] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.175360   521 consensus_queue.cc:260] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
W20260709 12:17:23.175354   447 leader_election.cc:343] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.177541   521 ts_tablet_manager.cc:1434] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:17:23.178792   521 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: Bootstrap starting.
I20260709 12:17:23.178930   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.179795   407 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.189236   521 tablet_bootstrap.cc:654] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.189713 32695 catalog_manager.cc:5697] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 1 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:23.192489   539 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.193192   539 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.197765   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:23.199153   539 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:23.202826   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
W20260709 12:17:23.200502   445 leader_election.cc:343] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.205706   521 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: No bootstrap required, opened a new log
I20260709 12:17:23.206229   521 ts_tablet_manager.cc:1403] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.003s
I20260709 12:17:23.209020   521 raft_consensus.cc:359] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
W20260709 12:17:23.209488   447 leader_election.cc:343] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.210153   447 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe; no voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b
I20260709 12:17:23.211692   539 raft_consensus.cc:2749] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:23.213649 32695 catalog_manager.cc:5697] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to efe9c93a145a490daba616e633c14a8b (127.21.150.193). New cstate: current_term: 1 leader_uuid: "efe9c93a145a490daba616e633c14a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.218258   555 leader_election.cc:290] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:23.219434   523 raft_consensus.cc:385] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.219980   523 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.223863   407 raft_consensus.cc:2468] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:23.225803   523 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.226327   445 leader_election.cc:304] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:23.227540   555 raft_consensus.cc:2804] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.228099   523 ts_tablet_manager.cc:1434] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.001s
I20260709 12:17:23.228190   555 raft_consensus.cc:697] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:23.229331   555 consensus_queue.cc:237] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.231880   522 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:23.232414   522 ts_tablet_manager.cc:1403] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.108s	user 0.047s	sys 0.041s
I20260709 12:17:23.229339   523 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:23.209712   521 raft_consensus.cc:385] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.233565   521 raft_consensus.cc:740] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.234248   521 consensus_queue.cc:260] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.235206   522 raft_consensus.cc:359] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.236289   521 ts_tablet_manager.cc:1434] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:17:23.235968   522 raft_consensus.cc:385] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.236932   522 raft_consensus.cc:740] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.237782   522 consensus_queue.cc:260] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.238243 32696 catalog_manager.cc:5697] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 1 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:23.239703   523 tablet_bootstrap.cc:654] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.246462   522 ts_tablet_manager.cc:1434] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 12:17:23.250104   522 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:23.250690   523 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:23.251274   523 ts_tablet_manager.cc:1403] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:17:23.254150   523 raft_consensus.cc:359] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.255285   523 raft_consensus.cc:385] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.255573   522 tablet_bootstrap.cc:654] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.255607   523 raft_consensus.cc:740] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.256524   523 consensus_queue.cc:260] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.258657   523 ts_tablet_manager.cc:1434] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:23.259545   523 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:23.262636   522 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:23.263176   522 ts_tablet_manager.cc:1403] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.003s
I20260709 12:17:23.264387   523 tablet_bootstrap.cc:654] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.265304   522 raft_consensus.cc:359] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.266343   522 raft_consensus.cc:385] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.266577   522 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.267222   522 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.268872   522 ts_tablet_manager.cc:1434] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:23.268900   523 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:23.269469   523 ts_tablet_manager.cc:1403] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.010s	user 0.005s	sys 0.003s
I20260709 12:17:23.270041   522 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: Bootstrap starting.
I20260709 12:17:23.272173   523 raft_consensus.cc:359] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.272750   523 raft_consensus.cc:385] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.272972   523 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.273571   555 raft_consensus.cc:493] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.273677   523 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.274111   555 raft_consensus.cc:515] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.275841   523 ts_tablet_manager.cc:1434] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:17:23.276310   555 leader_election.cc:290] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:23.276804   523 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:23.277169   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b" is_pre_election: true
I20260709 12:17:23.277884   332 raft_consensus.cc:2468] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 0.
I20260709 12:17:23.279172   447 leader_election.cc:304] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.281422   555 raft_consensus.cc:2804] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.281834   555 raft_consensus.cc:493] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.282141   555 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.282335   522 tablet_bootstrap.cc:654] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.282971   523 tablet_bootstrap.cc:654] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:23.287070   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
W20260709 12:17:23.289073   445 leader_election.cc:343] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:23.293928   523 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: No bootstrap required, opened a new log
I20260709 12:17:23.293864   555 raft_consensus.cc:515] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.294559   523 ts_tablet_manager.cc:1403] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 12:17:23.296893   332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:23.297495   332 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.297223   523 raft_consensus.cc:359] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.297935   523 raft_consensus.cc:385] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.298214   523 raft_consensus.cc:740] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.299010   522 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: No bootstrap required, opened a new log
I20260709 12:17:23.298885   523 consensus_queue.cc:260] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.299393   522 ts_tablet_manager.cc:1403] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.011s
I20260709 12:17:23.301062   523 ts_tablet_manager.cc:1434] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:17:23.302161   522 raft_consensus.cc:359] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.303323   522 raft_consensus.cc:385] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:23.303602   522 raft_consensus.cc:740] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Initialized, Role: FOLLOWER
I20260709 12:17:23.304312   522 consensus_queue.cc:260] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.304821   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.305490   407 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.307250   522 ts_tablet_manager.cc:1434] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:17:23.306981   332 raft_consensus.cc:2468] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:23.308540   447 leader_election.cc:304] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.309376   539 raft_consensus.cc:2804] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.309881   539 raft_consensus.cc:697] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:23.310699   539 consensus_queue.cc:237] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:23.313221   407 raft_consensus.cc:2468] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:23.314684   555 leader_election.cc:290] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 1 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:23.319864 32695 catalog_manager.cc:5697] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 1 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.357129   545 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.357707   545 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.359385   545 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.360407   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:23.360613   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:23.361069   407 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.361307   481 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.362155 32764 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.362973   545 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.363324   545 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.363716   545 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.369473   545 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.371245   545 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Requested vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.372277   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.372431   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:23.372843   407 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.373055   481 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.378152   481 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.378327   407 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.379211 32764 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.379879   545 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.380292   545 raft_consensus.cc:697] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Becoming Leader. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Running, Role: LEADER
I20260709 12:17:23.381105   545 consensus_queue.cc:237] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.388091 32696 catalog_manager.cc:5697] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to efe9c93a145a490daba616e633c14a8b (127.21.150.193). New cstate: current_term: 1 leader_uuid: "efe9c93a145a490daba616e633c14a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.421205   545 raft_consensus.cc:493] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.421890   545 raft_consensus.cc:515] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.423520   545 leader_election.cc:290] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.424459   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:23.424683   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:23.425139   407 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.425228   481 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.426239 32764 leader_election.cc:304] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.426910   545 raft_consensus.cc:2804] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.427240   545 raft_consensus.cc:493] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.427513   545 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.432977   545 raft_consensus.cc:515] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.434561   545 leader_election.cc:290] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Requested vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.435372   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.435617   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:23.435900   407 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.436214   481 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.441236   407 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.441344   481 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.442531 32764 leader_election.cc:304] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.443310   545 raft_consensus.cc:2804] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.443773   545 raft_consensus.cc:697] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Becoming Leader. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Running, Role: LEADER
I20260709 12:17:23.444440   545 consensus_queue.cc:237] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.451419 32696 catalog_manager.cc:5697] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to efe9c93a145a490daba616e633c14a8b (127.21.150.193). New cstate: current_term: 1 leader_uuid: "efe9c93a145a490daba616e633c14a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.455801   527 consensus_queue.cc:1048] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.468031   557 consensus_queue.cc:1048] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:23.473119   332 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.482753   539 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.488862   545 raft_consensus.cc:493] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.489367   545 raft_consensus.cc:515] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.491482   545 leader_election.cc:290] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.492592   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:23.493371   407 raft_consensus.cc:2393] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efe9c93a145a490daba616e633c14a8b in current term 1: Already voted for candidate 784b9febfff245dda6a27c4be73884fe in this term.
I20260709 12:17:23.495440   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:23.495838   555 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.497164 32764 leader_election.cc:304] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe9c93a145a490daba616e633c14a8b; no voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:23.498912   528 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.504817   481 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.506731   528 raft_consensus.cc:2749] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:23.508322   555 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.515688   528 consensus_queue.cc:1048] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:23.535765   332 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.537925   545 consensus_queue.cc:1048] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:23.545032   529 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.586905   528 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:23.596237   545 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:23.596738   545 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.597507   563 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:23.598886   545 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.600800   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:23.602277   481 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.601132   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:23.603417   406 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 0.
I20260709 12:17:23.603760 32764 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.605325   528 raft_consensus.cc:2804] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:23.605767   528 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:23.606185   528 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.613490   528 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.616220   528 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Requested vote from peers 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:23.617043   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:23.617497   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "efe9c93a145a490daba616e633c14a8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:23.617724   407 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.618100   481 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:23.623818   407 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.624686   481 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe9c93a145a490daba616e633c14a8b in term 1.
I20260709 12:17:23.625030 32764 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:23.625833   528 raft_consensus.cc:2804] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:23.626260   528 raft_consensus.cc:697] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Becoming Leader. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Running, Role: LEADER
I20260709 12:17:23.627177   528 consensus_queue.cc:237] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:23.635363 32697 catalog_manager.cc:5697] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b reported cstate change: term changed from 0 to 1, leader changed from <none> to efe9c93a145a490daba616e633c14a8b (127.21.150.193). New cstate: current_term: 1 leader_uuid: "efe9c93a145a490daba616e633c14a8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:23.731182 22107 ts_manager.cc:295] Set tserver state for efe9c93a145a490daba616e633c14a8b to MAINTENANCE_MODE
I20260709 12:17:23.731995 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
W20260709 12:17:23.754961   447 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:23.759202 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:23.760135 22107 tablet_replica.cc:333] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.760774 22107 raft_consensus.cc:2243] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:23.761227 22107 raft_consensus.cc:2272] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.763121 22107 tablet_replica.cc:333] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.763698 22107 raft_consensus.cc:2243] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:23.764289 22107 pending_rounds.cc:70] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Trying to abort 1 pending ops.
I20260709 12:17:23.764492 22107 pending_rounds.cc:77] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623321104429056 op_type: NO_OP noop_request { }
I20260709 12:17:23.764802 22107 raft_consensus.cc:2889] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:23.765081 22107 raft_consensus.cc:2272] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.767428 22107 tablet_replica.cc:333] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.768064 22107 raft_consensus.cc:2243] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:23.768479 22107 raft_consensus.cc:2272] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.770617 22107 tablet_replica.cc:333] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.771178 22107 raft_consensus.cc:2243] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:23.771749 22107 pending_rounds.cc:70] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Trying to abort 1 pending ops.
I20260709 12:17:23.771955 22107 pending_rounds.cc:77] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623320355942400 op_type: NO_OP noop_request { }
I20260709 12:17:23.772233 22107 raft_consensus.cc:2889] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:23.772503 22107 raft_consensus.cc:2272] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.774663 22107 tablet_replica.cc:333] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.775203 22107 raft_consensus.cc:2243] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:23.775789 22107 pending_rounds.cc:70] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Trying to abort 1 pending ops.
I20260709 12:17:23.776011 22107 pending_rounds.cc:77] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623320096473088 op_type: NO_OP noop_request { }
I20260709 12:17:23.776280 22107 raft_consensus.cc:2889] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:23.776571 22107 raft_consensus.cc:2272] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.778698 22107 tablet_replica.cc:333] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.779443 22107 raft_consensus.cc:2243] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:23.779845 22107 raft_consensus.cc:2272] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.779495   555 consensus_queue.cc:1048] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:23.781867 22107 tablet_replica.cc:333] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.782588 22107 raft_consensus.cc:2243] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:23.783010 22107 raft_consensus.cc:2272] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.785676 22107 tablet_replica.cc:333] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.786209 22107 raft_consensus.cc:2243] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:23.786649 22107 raft_consensus.cc:2272] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.789768 22107 tablet_replica.cc:333] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.790467 22107 raft_consensus.cc:2243] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:23.791472 22107 raft_consensus.cc:2272] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.795894 22107 tablet_replica.cc:333] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: stopping tablet replica
I20260709 12:17:23.796392 22107 raft_consensus.cc:2243] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:23.797166 22107 raft_consensus.cc:2272] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:23.817534 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
W20260709 12:17:23.820498 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:23.830210   447 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:23.840446   555 consensus_queue.cc:1048] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:23.910703   373 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.050882   447 consensus_peers.cc:597] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.175086   447 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:24.821713 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:24.917119   527 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:24.917800   527 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:24.919670   527 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:24.920866   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:24.921571   481 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
I20260709 12:17:24.922623   371 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:24.923413   527 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:24.923439   373 leader_election.cc:336] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:24.923914   527 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:24.924273   527 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:24.929859   527 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:24.931702   527 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:24.932955   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:24.933611   481 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:24.935252   373 leader_election.cc:336] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:24.939491   481 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:24.940603   371 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:24.941318   527 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:24.942214   527 raft_consensus.cc:697] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:24.942895   527 consensus_queue.cc:237] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:24.949661 32697 catalog_manager.cc:5697] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from efe9c93a145a490daba616e633c14a8b (127.21.150.193) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:25.044605   527 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
I20260709 12:17:25.045167   527 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.046952   527 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.048128   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:25.048322   529 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
W20260709 12:17:25.048506   373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111) [suppressed 114 similar messages]
I20260709 12:17:25.049008   529 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.049958   481 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
W20260709 12:17:25.051614   373 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.052733   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:25.053318   371 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.053501   529 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:25.053416   407 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:25.054251   527 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.054591   527 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
I20260709 12:17:25.054983   527 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.055148   445 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:25.056077   529 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.056463   529 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
W20260709 12:17:25.056676   447 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.056854   529 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.060614   527 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.062419   527 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.062356   529 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.063565   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:25.064411   481 raft_consensus.cc:2393] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7dde1735325f414bbdb50b608d80dca1 in current term 2: Already voted for candidate 784b9febfff245dda6a27c4be73884fe in this term.
I20260709 12:17:25.065418   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:25.066213   407 raft_consensus.cc:2393] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 784b9febfff245dda6a27c4be73884fe in current term 2: Already voted for candidate 7dde1735325f414bbdb50b608d80dca1 in this term.
W20260709 12:17:25.066931   373 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.067408   373 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1; no voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.065889   529 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:25.068200   527 raft_consensus.cc:2749] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:17:25.068758   447 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.069187   447 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe; no voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.069799   529 raft_consensus.cc:2749] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:17:25.095796   529 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
I20260709 12:17:25.096426   555 raft_consensus.cc:493] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.096396   529 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.097013   555 raft_consensus.cc:515] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.098599   529 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:25.099179   555 leader_election.cc:290] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:25.099665   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:25.100711   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1" is_pre_election: true
I20260709 12:17:25.101349   406 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 1.
I20260709 12:17:25.102349   527 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
I20260709 12:17:25.102636   445 leader_election.cc:304] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:25.102994   527 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
W20260709 12:17:25.103448   447 leader_election.cc:336] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.103528   555 raft_consensus.cc:2804] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:17:25.103994   447 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe; no voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.104029   555 raft_consensus.cc:493] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.104571   555 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.104736   529 raft_consensus.cc:2749] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:17:25.104967   447 leader_election.cc:336] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.105952   527 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.107123   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:25.107767   481 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
I20260709 12:17:25.108675   371 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:25.109450   527 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:25.109474   373 leader_election.cc:336] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.109900   527 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efe9c93a145a490daba616e633c14a8b)
I20260709 12:17:25.110328   527 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.111833   555 raft_consensus.cc:515] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.113634   555 leader_election.cc:290] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:25.114866   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:25.115525   406 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.116396   527 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
W20260709 12:17:25.117627   447 leader_election.cc:336] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.118637   527 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.120570   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:25.121179   481 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:25.122098   373 leader_election.cc:336] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.126478   406 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 2.
I20260709 12:17:25.127118   481 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:25.127812   445 leader_election.cc:304] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.128208   371 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.128602   555 raft_consensus.cc:2804] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:25.129020   527 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:25.129136   555 raft_consensus.cc:697] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:25.129626   527 raft_consensus.cc:697] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:25.130106   555 consensus_queue.cc:237] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.130491   527 consensus_queue.cc:237] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.140002 32697 catalog_manager.cc:5697] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 1 to 2, leader changed from efe9c93a145a490daba616e633c14a8b (127.21.150.193) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 2 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:25.141523 32696 catalog_manager.cc:5697] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from efe9c93a145a490daba616e633c14a8b (127.21.150.193) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:25.417302   481 raft_consensus.cc:1275] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:25.418831   527 consensus_queue.cc:1048] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:25.433109   373 consensus_peers.cc:597] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.481398   527 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:25.481954   527 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.483590   527 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.484763   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:25.485517   481 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 1.
I20260709 12:17:25.486503   371 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:25.487210   527 raft_consensus.cc:2804] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:17:25.487347   373 leader_election.cc:336] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.487540   527 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:25.487937   527 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:25.492865   527 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.494479   527 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:25.495568   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:25.496199   481 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:17:25.498050   373 leader_election.cc:336] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:25.501679   481 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:25.502614   371 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:25.503336   527 raft_consensus.cc:2804] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:25.503744   527 raft_consensus.cc:697] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:25.504383   527 consensus_queue.cc:237] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:25.512100 32696 catalog_manager.cc:5697] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from efe9c93a145a490daba616e633c14a8b (127.21.150.193) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:25.594156   406 raft_consensus.cc:1275] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:25.595505   555 consensus_queue.cc:1048] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:25.609513   447 consensus_peers.cc:597] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.713898   481 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:25.715126   527 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:25.726785   373 consensus_peers.cc:597] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:25.822654 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:25.957005   373 consensus_peers.cc:597] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:25.959218   481 raft_consensus.cc:1275] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:17:25.960446   527 consensus_queue.cc:1048] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:17:25.988526   373 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:26.228653   447 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:26.398542   447 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:26.692428   447 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:26.823784 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:26.964049   447 consensus_peers.cc:597] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:17:27.205634   609 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:27.206396   609 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:27.208577   609 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:27.210320   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe" is_pre_election: true
I20260709 12:17:27.211541   481 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:27.212999   371 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
W20260709 12:17:27.213653   373 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:27.213843   609 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:27.214314   609 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:27.214607   609 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:27.222710   609 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:27.224444   609 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:27.225692   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:27.226336   481 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:27.228327   373 leader_election.cc:336] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111)
I20260709 12:17:27.234553   481 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 3.
I20260709 12:17:27.236083   371 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: efe9c93a145a490daba616e633c14a8b
I20260709 12:17:27.236920   609 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:27.237388   609 raft_consensus.cc:697] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:27.238503   609 consensus_queue.cc:237] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:27.255254 32697 catalog_manager.cc:5697] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 3, leader changed from efe9c93a145a490daba616e633c14a8b (127.21.150.193) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:27.785010   481 raft_consensus.cc:1275] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:27.786234   609 consensus_queue.cc:1048] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:27.803109   373 consensus_peers.cc:597] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:27.825057 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:27.906564   373 consensus_peers.cc:597] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:28.083719   373 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:28.133980   447 consensus_peers.cc:597] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:28.322634   373 consensus_peers.cc:597] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:28.342118   373 consensus_peers.cc:597] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:28.536996   447 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:28.826071 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:28.838336 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:28.846206   640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:28.846767   641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:28.848397   643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:28.850397 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:28.852149 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:28.852352 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:28.852556 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599448852542 us; error 0 us; skew 500 ppm
I20260709 12:17:28.853231 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:28.856267 22107 webserver.cc:533] Webserver started at http://127.21.150.193:33769/ using document root <none> and password file <none>
I20260709 12:17:28.856819 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:28.857033 22107 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:17:28.861081 22107 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:28.865075   648 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:28.865912 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:28.866225 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "efe9c93a145a490daba616e633c14a8b"
format_stamp: "Formatted at 2026-07-09 12:17:21 on dist-test-slave-zctp"
I20260709 12:17:28.866588 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:28.880314 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:28.881712 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:28.883769 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:28.888042   655 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 12:17:28.943120 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:17:28.943440 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.056s	user 0.002s	sys 0.000s
I20260709 12:17:28.943745 22107 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:17:28.949024   655 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:28.962083   655 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:28.963153   655 tablet_bootstrap.cc:492] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:28.963812   655 ts_tablet_manager.cc:1403] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:17:28.966475   655 raft_consensus.cc:359] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:28.967224   655 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:28.967846   655 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:28.970482   655 ts_tablet_manager.cc:1434] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:28.971194   655 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:28.983213 22107 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:17:28.983482 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.040s	user 0.035s	sys 0.004s
I20260709 12:17:28.983945   655 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:28.984747   655 tablet_bootstrap.cc:492] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:28.985342   655 ts_tablet_manager.cc:1403] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:17:28.987820   655 raft_consensus.cc:359] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:28.988373   655 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:28.989073   655 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:28.990727   655 ts_tablet_manager.cc:1434] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:28.991513   655 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.004483   655 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.005203   655 tablet_bootstrap.cc:492] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.005985   655 ts_tablet_manager.cc:1403] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:17:29.009080   655 raft_consensus.cc:359] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.009740   655 raft_consensus.cc:740] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.010529   655 consensus_queue.cc:260] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.012297   655 ts_tablet_manager.cc:1434] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:17:29.012988   655 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.027695   655 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.028769   655 tablet_bootstrap.cc:492] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.029548   655 ts_tablet_manager.cc:1403] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:17:29.031565   655 raft_consensus.cc:359] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.032091   655 raft_consensus.cc:740] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.032750   655 consensus_queue.cc:260] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.034401   655 ts_tablet_manager.cc:1434] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:29.035092   655 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.048573   655 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.049381   655 tablet_bootstrap.cc:492] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.049980   655 ts_tablet_manager.cc:1403] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:17:29.050240 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:44501
I20260709 12:17:29.050377   717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:44501 every 8 connection(s)
I20260709 12:17:29.052248   655 raft_consensus.cc:359] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.052825   655 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.053512   655 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.055301   655 ts_tablet_manager.cc:1434] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 12:17:29.055881 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:29.057767   655 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.077759   718 heartbeater.cc:344] Connected to a master server at 127.21.150.254:43431
I20260709 12:17:29.078441   718 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:29.079720   718 heartbeater.cc:507] Master 127.21.150.254:43431 requested a full tablet report, sending...
I20260709 12:17:29.088548 32696 ts_manager.cc:194] Re-registered known tserver with Master: efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501)
I20260709 12:17:29.089718   655 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:29.090783   655 tablet_bootstrap.cc:492] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.091542   655 ts_tablet_manager.cc:1403] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.003s
I20260709 12:17:29.097587   655 raft_consensus.cc:359] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.101365   655 raft_consensus.cc:740] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.102310   655 consensus_queue.cc:260] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.104718 32696 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40708
I20260709 12:17:29.107448   655 ts_tablet_manager.cc:1434] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
W20260709 12:17:29.107995   447 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:17:29.108418   655 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.109231   718 heartbeater.cc:499] Master 127.21.150.254:43431 was elected leader, sending a full tablet report...
I20260709 12:17:29.116525   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:29.117235   481 raft_consensus.cc:606] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:29.123397   655 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:17:29.124505   655 tablet_bootstrap.cc:492] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.125159   655 ts_tablet_manager.cc:1403] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:17:29.127462   655 raft_consensus.cc:359] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.128192   655 raft_consensus.cc:740] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.128986   655 consensus_queue.cc:260] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.130945   655 ts_tablet_manager.cc:1434] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:29.131006   406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:29.131651   406 raft_consensus.cc:606] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:29.131762   655 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.134678   406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:29.135260   406 raft_consensus.cc:606] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:29.137907 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:29.138460 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:29.139142 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:29.147295   655 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.148806   655 tablet_bootstrap.cc:492] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.149638   655 ts_tablet_manager.cc:1403] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.007s
I20260709 12:17:29.152686   655 raft_consensus.cc:359] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.153465   655 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.154230   655 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.156040   655 ts_tablet_manager.cc:1434] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:17:29.156701   655 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.163550   692 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.168525   655 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.169327   655 tablet_bootstrap.cc:492] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.169939   655 ts_tablet_manager.cc:1403] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:17:29.170761   692 pending_rounds.cc:85] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Aborting all ops after (but not including) 0
I20260709 12:17:29.171068   692 pending_rounds.cc:107] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:17:29.171309   692 raft_consensus.cc:2889] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.172039   655 raft_consensus.cc:359] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:29.172642   655 raft_consensus.cc:740] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.173271   655 consensus_queue.cc:260] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:29.175623   655 ts_tablet_manager.cc:1434] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:17:29.176501   655 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Bootstrap starting.
I20260709 12:17:29.193897   655 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:17:29.194656   655 tablet_bootstrap.cc:492] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Bootstrap complete.
I20260709 12:17:29.195211   655 ts_tablet_manager.cc:1403] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 12:17:29.197120   655 raft_consensus.cc:359] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.197677   655 raft_consensus.cc:740] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe9c93a145a490daba616e633c14a8b, State: Initialized, Role: FOLLOWER
I20260709 12:17:29.198333   655 consensus_queue.cc:260] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.199785   655 ts_tablet_manager.cc:1434] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:17:29.237694   609 raft_consensus.cc:993] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:29.238273   734 raft_consensus.cc:1081] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:29.239804   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:29.240355   481 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:29.240635   481 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.245579   481 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.247223   481 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:29.248172   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:29.248329   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:29.248808   692 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.248932   406 raft_consensus.cc:3055] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.249317   406 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:29.250171   406 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.251220   406 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.254401   692 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:29.255663   691 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.255662   447 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:29.256852   639 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:29.257972   406 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:29.258503   639 raft_consensus.cc:697] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:29.259758   639 consensus_queue.cc:237] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.269093 32696 catalog_manager.cc:5697] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 2 to 3, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 3 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:29.276539   735 raft_consensus.cc:993] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:29.277053   639 raft_consensus.cc:1081] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:29.278880   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:29.279527   406 raft_consensus.cc:493] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:29.279850   406 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.285990   406 raft_consensus.cc:515] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.287530   406 leader_election.cc:290] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:29.288393   691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:29.288545   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:29.289172   691 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.289188   481 raft_consensus.cc:3055] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:29.289765   481 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:29.290647   481 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.291996   481 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.296952   691 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:29.298166   373 leader_election.cc:304] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:29.299032   734 raft_consensus.cc:2804] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:29.299333   481 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:29.301558   734 raft_consensus.cc:697] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:29.302857   734 consensus_queue.cc:237] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.308663   609 raft_consensus.cc:993] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:29.309190   609 raft_consensus.cc:1081] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:29.310923   480 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:29.311506   480 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:29.311230 32696 catalog_manager.cc:5697] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:29.311991   480 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.320395   480 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.322751   480 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:29.324415   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:29.325121   407 raft_consensus.cc:3055] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:29.325018   691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:29.325748   407 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:29.326015   691 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.326749   407 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.327883   407 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:29.332361   691 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:29.333768   447 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:29.334640   639 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:29.335126   407 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:29.335155   639 raft_consensus.cc:697] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:29.336488   639 consensus_queue.cc:237] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:29.344287 32696 catalog_manager.cc:5697] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 2 to 3, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 3 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:29.508549   691 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:29.513998   691 pending_rounds.cc:85] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Aborting all ops after (but not including) 0
I20260709 12:17:29.514268   691 pending_rounds.cc:107] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:17:29.514499   691 raft_consensus.cc:2889] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.641568   691 raft_consensus.cc:1275] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.642745   639 consensus_queue.cc:1048] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:29.643201   639 consensus_queue.cc:1243] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [LEADER]: Peer efe9c93a145a490daba616e633c14a8b log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 12:17:29.645084   691 pending_rounds.cc:85] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Aborting all ops after (but not including) 0
I20260709 12:17:29.645293   691 pending_rounds.cc:107] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:17:29.645539   691 raft_consensus.cc:2889] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:29.655802   407 raft_consensus.cc:1275] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:29.657302   746 consensus_queue.cc:1048] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:29.697641   692 raft_consensus.cc:1275] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:29.698907   734 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.708144   480 raft_consensus.cc:1275] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:29.710098   609 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.807663   692 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:29.809091   746 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:29.818701   407 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:29.821290   639 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:17:29.827080 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:30.147763 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:30.157984   407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:30.158602   407 raft_consensus.cc:606] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:30.160912   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:30.161314   481 raft_consensus.cc:606] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:30.163498   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:30.163861   481 raft_consensus.cc:606] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:30.164732 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:30.165233 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:30.165823 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:30.262154   746 raft_consensus.cc:993] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:30.262676   749 raft_consensus.cc:1081] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:30.264015   407 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:30.264529   407 raft_consensus.cc:493] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:30.264851   407 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.270290   407 raft_consensus.cc:515] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.272032   407 leader_election.cc:290] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:30.273041   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:30.273619   481 raft_consensus.cc:3055] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:30.273671   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:30.273886   481 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:30.274322   692 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.274770   481 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.275744   481 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.279354   692 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:30.280297   481 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:30.280565   373 leader_election.cc:304] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:30.281383   609 raft_consensus.cc:2804] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:30.281838   609 raft_consensus.cc:697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:30.282691   609 consensus_queue.cc:237] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.289047 32696 catalog_manager.cc:5697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:30.300170   746 raft_consensus.cc:993] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:30.300575   749 raft_consensus.cc:1081] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:30.301962   407 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:30.302433   407 raft_consensus.cc:493] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:30.302718   407 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.308182   407 raft_consensus.cc:515] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:30.309962   407 leader_election.cc:290] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:30.310655   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:30.310925   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a761e98905764ff5af3aa30229ca50d4" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:30.311265   692 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.311534   481 raft_consensus.cc:3055] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:30.311846   481 raft_consensus.cc:740] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:30.312453   481 consensus_queue.cc:260] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:30.313388   481 raft_consensus.cc:3060] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:30.316596   692 raft_consensus.cc:2468] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:30.317786   373 leader_election.cc:304] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:30.318676   609 raft_consensus.cc:2804] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:30.318682   481 raft_consensus.cc:2468] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:30.319209   609 raft_consensus.cc:697] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:30.320436   609 consensus_queue.cc:237] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } }
I20260709 12:17:30.327615 32696 catalog_manager.cc:5697] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:30.708652   734 raft_consensus.cc:993] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:30.709283   734 raft_consensus.cc:1081] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:30.712108   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:30.712735   481 raft_consensus.cc:493] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:30.713157   481 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.718566   481 raft_consensus.cc:515] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.720052   481 leader_election.cc:290] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:30.720813   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:30.721035   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49b3b5891cbb48d19e8dead71573964b" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:30.721511   692 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.721657   407 raft_consensus.cc:3055] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:30.722038   407 raft_consensus.cc:740] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:30.722687   407 consensus_queue.cc:260] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.723560   407 raft_consensus.cc:3060] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:30.726965   692 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:30.728125   447 leader_election.cc:304] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:30.728813   749 raft_consensus.cc:2804] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:30.728785   407 raft_consensus.cc:2468] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:30.729508   749 raft_consensus.cc:697] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:30.730291   749 consensus_queue.cc:237] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:30.737176 32696 catalog_manager.cc:5697] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 2 to 3, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 3 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:30.738215   481 raft_consensus.cc:1275] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.739630   734 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:30.741946   692 raft_consensus.cc:1275] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.743328   734 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:17:30.828174 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:30.846074   691 raft_consensus.cc:1275] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.847601   734 consensus_queue.cc:1048] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:30.856204   481 raft_consensus.cc:1275] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:30.858561   762 consensus_queue.cc:1048] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.173043 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:31.180696   407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:31.181322   407 raft_consensus.cc:606] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:31.184283   407 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:31.184749   407 raft_consensus.cc:606] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:31.187075   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d249715915f64545ba53f74e84a294e4"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:31.187554   481 raft_consensus.cc:606] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:31.188704 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:31.189167 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:31.189708 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:31.203023   609 raft_consensus.cc:993] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:31.203657   762 raft_consensus.cc:1081] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:31.205206   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:31.205657   481 raft_consensus.cc:493] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:31.205932   481 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.210732   691 raft_consensus.cc:1275] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.212013   749 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.212596   481 raft_consensus.cc:515] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.214987   481 leader_election.cc:290] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:31.215304   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:31.215663   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaae3f0f5bc646a6aeb204604f7d99d3" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:31.215926   692 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.216253   407 raft_consensus.cc:3055] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:31.216552   407 raft_consensus.cc:740] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:31.217285   407 consensus_queue.cc:260] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.220857   407 raft_consensus.cc:3060] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.223842   692 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:31.225292   447 leader_election.cc:304] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:31.226287   749 raft_consensus.cc:2804] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:31.226835   749 raft_consensus.cc:697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:31.227674   749 consensus_queue.cc:237] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.231772   406 raft_consensus.cc:1275] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.234827   639 consensus_queue.cc:1048] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.245343   407 raft_consensus.cc:2468] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:31.249080 32697 catalog_manager.cc:5697] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 2 to 3, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 3 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:31.388856   773 raft_consensus.cc:993] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:31.389382   749 raft_consensus.cc:1081] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:31.390964   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d249715915f64545ba53f74e84a294e4"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:31.391408   406 raft_consensus.cc:493] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:31.391711   406 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.397568   406 raft_consensus.cc:515] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.400017   406 leader_election.cc:290] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:31.401178   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:31.401209   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d249715915f64545ba53f74e84a294e4" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:31.402479   481 raft_consensus.cc:3055] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:31.402832   481 raft_consensus.cc:740] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:31.402908   692 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.403703   481 consensus_queue.cc:260] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.405040   481 raft_consensus.cc:3060] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:31.410627   692 raft_consensus.cc:2468] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:31.411326   734 raft_consensus.cc:993] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:31.411829   734 raft_consensus.cc:1081] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:31.411897   481 raft_consensus.cc:2468] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 2.
I20260709 12:17:31.413619   371 leader_election.cc:304] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, 7dde1735325f414bbdb50b608d80dca1; no voters: 
I20260709 12:17:31.413885   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:31.414512   481 raft_consensus.cc:493] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:31.414535   762 raft_consensus.cc:2804] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:31.414945   481 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.415629   762 raft_consensus.cc:697] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:31.416615   762 consensus_queue.cc:237] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.420964   481 raft_consensus.cc:515] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.423460   481 leader_election.cc:290] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:31.423740   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:31.424022   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56b6194d92c34c089bf0c6a8ecef5930" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:31.424396   692 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.424654   406 raft_consensus.cc:3055] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:31.424983   406 raft_consensus.cc:740] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:31.426038   406 consensus_queue.cc:260] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.426286 32697 catalog_manager.cc:5697] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 1 to 2, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:31.427397   406 raft_consensus.cc:3060] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:31.431952   692 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:31.433223   447 leader_election.cc:304] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:31.434026   749 raft_consensus.cc:2804] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:31.434425   749 raft_consensus.cc:697] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:31.435009   406 raft_consensus.cc:2468] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 3.
I20260709 12:17:31.435159   749 consensus_queue.cc:237] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:31.443152 32697 catalog_manager.cc:5697] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 2 to 3, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 3 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:31.829315 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:31.844833   692 raft_consensus.cc:1275] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.846251   749 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.862316   406 raft_consensus.cc:1275] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:31.863695   774 consensus_queue.cc:1048] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:31.892591   692 raft_consensus.cc:1275] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:31.893836   762 consensus_queue.cc:1048] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.902712   481 raft_consensus.cc:1275] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:31.904839   609 consensus_queue.cc:1048] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.925246   692 raft_consensus.cc:1275] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:31.926498   774 consensus_queue.cc:1048] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:31.935089   406 raft_consensus.cc:1275] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:31.936609   774 consensus_queue.cc:1048] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:32.196079 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:32.203598   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:32.204165   481 raft_consensus.cc:606] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:32.206667   406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:32.207175   406 raft_consensus.cc:606] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:32.209586   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:32.210028   481 raft_consensus.cc:606] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:32.210948 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:32.211447 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:32.212009 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:32.280483   773 raft_consensus.cc:993] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:32.281046   774 raft_consensus.cc:1081] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:32.282503   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:32.282946   406 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:32.283192   406 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.288482   406 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.290344   406 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 4 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:32.291471   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:32.292050   481 raft_consensus.cc:3055] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.292341   481 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:32.291391   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:32.293025   692 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.293318   481 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.294636   481 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.300663   692 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 4.
I20260709 12:17:32.301678   481 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 4.
I20260709 12:17:32.302037   373 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:32.302884   609 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:32.303297   609 raft_consensus.cc:697] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:32.304034   609 consensus_queue.cc:237] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.311760 32696 catalog_manager.cc:5697] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 3 to 4, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:32.558302   749 raft_consensus.cc:993] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:32.558789   774 raft_consensus.cc:1081] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:32.560221   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:32.560648   406 raft_consensus.cc:493] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:32.561033   406 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.566162   406 raft_consensus.cc:515] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.567473   406 leader_election.cc:290] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 4 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:32.568473   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:32.569159   692 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.568539   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58d0d9e0f0f949099570bddfa5106e8f" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:32.569942   481 raft_consensus.cc:3055] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:32.570236   481 raft_consensus.cc:740] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:32.570890   481 consensus_queue.cc:260] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.571946   481 raft_consensus.cc:3060] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:32.575690   692 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 4.
I20260709 12:17:32.576808   373 leader_election.cc:304] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:32.577538   609 raft_consensus.cc:2804] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:32.577572   481 raft_consensus.cc:2468] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 4.
I20260709 12:17:32.578109   609 raft_consensus.cc:697] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:32.579010   609 consensus_queue.cc:237] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.585320 32696 catalog_manager.cc:5697] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 3 to 4, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 4 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:32.691449   796 raft_consensus.cc:993] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:32.691941   796 raft_consensus.cc:1081] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:32.693313   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:32.693768   481 raft_consensus.cc:493] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:32.694070   481 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.699342   481 raft_consensus.cc:515] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.700644   481 leader_election.cc:290] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:32.701507   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:32.701632   406 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75e138f0842544c9903c3e030689ccb7" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:32.702280   692 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.702368   406 raft_consensus.cc:3055] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:32.702745   406 raft_consensus.cc:740] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:32.703431   406 consensus_queue.cc:260] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.704319   406 raft_consensus.cc:3060] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:32.707072   692 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 2.
I20260709 12:17:32.708178   447 leader_election.cc:304] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:32.708940   774 raft_consensus.cc:2804] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:32.709347   774 raft_consensus.cc:697] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:32.709373   406 raft_consensus.cc:2468] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 2.
I20260709 12:17:32.710255   774 consensus_queue.cc:237] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:32.718017 32696 catalog_manager.cc:5697] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 1 to 2, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 2 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:32.828886   481 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:17:32.830319 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:32.830294   796 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:32.840766   692 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:17:32.842401   762 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:33.174330   692 raft_consensus.cc:1275] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.175769   609 consensus_queue.cc:1048] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.178242   481 raft_consensus.cc:1275] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.179757   762 consensus_queue.cc:1048] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.218969 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:33.227533   481 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
mode: GRACEFUL
new_leader_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:54668
I20260709 12:17:33.228046   481 raft_consensus.cc:606] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Received request to transfer leadership to TS 7dde1735325f414bbdb50b608d80dca1
I20260709 12:17:33.231011   406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:33.231467   406 raft_consensus.cc:606] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:33.233924   406 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
mode: GRACEFUL
new_leader_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:48070
I20260709 12:17:33.234364   406 raft_consensus.cc:606] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Received request to transfer leadership to TS 784b9febfff245dda6a27c4be73884fe
I20260709 12:17:33.235324 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:17:33.235925 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:33.236570 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:33.237684   406 raft_consensus.cc:1275] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:33.238988   774 consensus_queue.cc:1048] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:33.251650   692 raft_consensus.cc:1275] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:33.252859   808 consensus_queue.cc:1048] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:33.321126   762 raft_consensus.cc:993] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:33.321657   796 raft_consensus.cc:1081] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:33.323204   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:33.323282   774 raft_consensus.cc:993] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: : Instructing follower 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:33.323678   481 raft_consensus.cc:493] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.323843   808 raft_consensus.cc:1081] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Signalling peer 7dde1735325f414bbdb50b608d80dca1 to start an election
I20260709 12:17:33.323987   481 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.325486   406 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5"
dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
 from {username='slave'} at 127.0.0.1:48058
I20260709 12:17:33.325992   406 raft_consensus.cc:493] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.326313   406 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.330782   481 raft_consensus.cc:515] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.332746   481 leader_election.cc:290] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 5 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:33.332981   406 raft_consensus.cc:515] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.333755   692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:33.333952   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2a3bb67994c4a9e804975daa26f99f6" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:33.334360   692 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.334518   407 raft_consensus.cc:3055] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:17:33.334878   407 raft_consensus.cc:740] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:33.335624   691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:33.335624   407 consensus_queue.cc:260] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.336203   481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ee1edff6824b32bb78ced08ffba7f5" candidate_uuid: "7dde1735325f414bbdb50b608d80dca1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "784b9febfff245dda6a27c4be73884fe"
I20260709 12:17:33.336452   691 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.336783   481 raft_consensus.cc:3055] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:33.335294   406 leader_election.cc:290] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 784b9febfff245dda6a27c4be73884fe (127.21.150.195:43377)
I20260709 12:17:33.337085   407 raft_consensus.cc:3060] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:17:33.337113   481 raft_consensus.cc:740] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:33.339392   481 consensus_queue.cc:260] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.340727   481 raft_consensus.cc:3060] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:33.341385   692 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 5.
I20260709 12:17:33.342631   447 leader_election.cc:304] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:33.343497   808 raft_consensus.cc:2804] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:17:33.344000   808 raft_consensus.cc:697] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 5 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:33.344094   407 raft_consensus.cc:2468] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 5.
I20260709 12:17:33.344861   808 consensus_queue.cc:237] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.347158   691 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 3.
I20260709 12:17:33.347188   481 raft_consensus.cc:2468] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dde1735325f414bbdb50b608d80dca1 in term 3.
I20260709 12:17:33.348711   373 leader_election.cc:304] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dde1735325f414bbdb50b608d80dca1, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:33.349572   796 raft_consensus.cc:2804] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:33.350061   796 raft_consensus.cc:697] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Becoming Leader. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:33.350909   796 consensus_queue.cc:237] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.351619 32696 catalog_manager.cc:5697] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 4 to 5, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 5 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:33.359122 32695 catalog_manager.cc:5697] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 reported cstate change: term changed from 2 to 3, leader changed from 784b9febfff245dda6a27c4be73884fe (127.21.150.195) to 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "7dde1735325f414bbdb50b608d80dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:33.392118   762 raft_consensus.cc:993] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: : Instructing follower 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:33.392591   796 raft_consensus.cc:1081] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Signalling peer 784b9febfff245dda6a27c4be73884fe to start an election
I20260709 12:17:33.394200   481 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903"
dest_uuid: "784b9febfff245dda6a27c4be73884fe"
 from {username='slave'} at 127.0.0.1:54664
I20260709 12:17:33.394737   481 raft_consensus.cc:493] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:33.395219   481 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.402571   481 raft_consensus.cc:515] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.404145   481 leader_election.cc:290] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 4 election: Requested vote from peers efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501), 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963)
I20260709 12:17:33.404958   691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "efe9c93a145a490daba616e633c14a8b"
I20260709 12:17:33.405177   407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84f4d76718f446c78e1feb938aefc903" candidate_uuid: "784b9febfff245dda6a27c4be73884fe" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7dde1735325f414bbdb50b608d80dca1"
I20260709 12:17:33.405586   691 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.405848   407 raft_consensus.cc:3055] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:33.406111   407 raft_consensus.cc:740] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7dde1735325f414bbdb50b608d80dca1, State: Running, Role: LEADER
I20260709 12:17:33.406636   407 consensus_queue.cc:260] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.407336   407 raft_consensus.cc:3060] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:33.411223   691 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 4.
I20260709 12:17:33.412343   447 leader_election.cc:304] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 784b9febfff245dda6a27c4be73884fe, efe9c93a145a490daba616e633c14a8b; no voters: 
I20260709 12:17:33.413093   808 raft_consensus.cc:2804] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:33.413218   407 raft_consensus.cc:2468] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 784b9febfff245dda6a27c4be73884fe in term 4.
I20260709 12:17:33.413622   808 raft_consensus.cc:697] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 4 LEADER]: Becoming Leader. State: Replica: 784b9febfff245dda6a27c4be73884fe, State: Running, Role: LEADER
I20260709 12:17:33.414616   808 consensus_queue.cc:237] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } }
I20260709 12:17:33.422082 32695 catalog_manager.cc:5697] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe reported cstate change: term changed from 3 to 4, leader changed from 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194) to 784b9febfff245dda6a27c4be73884fe (127.21.150.195). New cstate: current_term: 4 leader_uuid: "784b9febfff245dda6a27c4be73884fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:33.827248   481 raft_consensus.cc:1275] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:33.828495   796 consensus_queue.cc:1048] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "784b9febfff245dda6a27c4be73884fe" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 43377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:17:33.831504 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:17:33.839039   691 raft_consensus.cc:1275] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Refusing update from remote peer 7dde1735325f414bbdb50b608d80dca1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:17:33.840770   819 consensus_queue.cc:1048] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:33.885619   407 raft_consensus.cc:1275] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.886914   808 consensus_queue.cc:1048] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.897509   691 raft_consensus.cc:1275] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:33.899286   825 consensus_queue.cc:1048] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:33.923657   691 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 5 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:33.925091   825 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe9c93a145a490daba616e633c14a8b" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:33.934046   407 raft_consensus.cc:1275] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 5 FOLLOWER]: Refusing update from remote peer 784b9febfff245dda6a27c4be73884fe: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:17:33.936229   808 consensus_queue.cc:1048] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dde1735325f414bbdb50b608d80dca1" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 40963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:17:34.323978 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:34.586459 22107 tablet_server.cc:179] TabletServer@127.21.150.193:44501 shutting down...
W20260709 12:17:34.618500   373 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 84 similar messages]
W20260709 12:17:34.622237   373 consensus_peers.cc:597] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.623103   373 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.643585   447 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.672597   447 consensus_peers.cc:597] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:34.687489 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:34.688287 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.688958 22107 raft_consensus.cc:2243] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.689375 22107 raft_consensus.cc:2272] T 11ee1edff6824b32bb78ced08ffba7f5 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.692382 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.692979 22107 raft_consensus.cc:2243] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.693509 22107 raft_consensus.cc:2272] T 49b3b5891cbb48d19e8dead71573964b P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.695374 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.695932 22107 raft_consensus.cc:2243] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.696355 22107 raft_consensus.cc:2272] T 56b6194d92c34c089bf0c6a8ecef5930 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.698177 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.698746 22107 raft_consensus.cc:2243] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.699158 22107 raft_consensus.cc:2272] T 75e138f0842544c9903c3e030689ccb7 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.700968 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.701615 22107 raft_consensus.cc:2243] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.702056 22107 raft_consensus.cc:2272] T aaae3f0f5bc646a6aeb204604f7d99d3 P efe9c93a145a490daba616e633c14a8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.704551 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.705250 22107 raft_consensus.cc:2243] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.705765 22107 raft_consensus.cc:2272] T 84f4d76718f446c78e1feb938aefc903 P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.708244 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.708976 22107 raft_consensus.cc:2243] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.709563 22107 raft_consensus.cc:2272] T d249715915f64545ba53f74e84a294e4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.711426 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.711967 22107 raft_consensus.cc:2243] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.712419 22107 raft_consensus.cc:2272] T f2a3bb67994c4a9e804975daa26f99f6 P efe9c93a145a490daba616e633c14a8b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.715230 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.715858 22107 raft_consensus.cc:2243] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.716280 22107 raft_consensus.cc:2272] T a761e98905764ff5af3aa30229ca50d4 P efe9c93a145a490daba616e633c14a8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:34.718235 22107 tablet_replica.cc:333] stopping tablet replica
I20260709 12:17:34.718694 22107 raft_consensus.cc:2243] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:34.719158 22107 raft_consensus.cc:2272] T 58d0d9e0f0f949099570bddfa5106e8f P efe9c93a145a490daba616e633c14a8b [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:34.728878   447 consensus_peers.cc:597] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.742995   373 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.796352   373 consensus_peers.cc:597] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.832623 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:34.903770   447 consensus_peers.cc:597] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.908896   447 consensus_peers.cc:597] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:34.947647   447 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:35.323268   428 debug-util.cc:398] Leaking SignalData structure 0x7b08002008c0 after lost signal to thread 22110
W20260709 12:17:35.324361   428 debug-util.cc:398] Leaking SignalData structure 0x7b080029b500 after lost signal to thread 32732
W20260709 12:17:35.325569   428 debug-util.cc:398] Leaking SignalData structure 0x7b08002f35c0 after lost signal to thread 431
W20260709 12:17:35.326514   428 debug-util.cc:398] Leaking SignalData structure 0x7b0800124b80 after lost signal to thread 505
W20260709 12:17:35.598519   714 debug-util.cc:398] Leaking SignalData structure 0x7b0800057760 after lost signal to thread 22110
W20260709 12:17:35.599606   714 debug-util.cc:398] Leaking SignalData structure 0x7b08001063e0 after lost signal to thread 32732
W20260709 12:17:35.600728   714 debug-util.cc:398] Leaking SignalData structure 0x7b080013cf00 after lost signal to thread 431
W20260709 12:17:35.601754   714 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2d80 after lost signal to thread 505
W20260709 12:17:35.748008 22107 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 714)
W20260709 12:17:35.833902 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:36.630446   373 consensus_peers.cc:597] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:36.748621 22107 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 714)
W20260709 12:17:36.835244 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:36.848501   447 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.158234   373 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.206435   373 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.256520   447 consensus_peers.cc:597] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.358079   373 consensus_peers.cc:597] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.594499   447 consensus_peers.cc:597] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.611164   447 consensus_peers.cc:597] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.619840   447 consensus_peers.cc:597] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.661314   447 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:17:37.749455 22107 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 714)
W20260709 12:17:37.836990 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:37.922605 32729 debug-util.cc:398] Leaking SignalData structure 0x7b08003fb240 after lost signal to thread 22110
W20260709 12:17:37.924853 32729 debug-util.cc:398] Leaking SignalData structure 0x7b0800036ac0 after lost signal to thread 32732
W20260709 12:17:37.926541 32729 debug-util.cc:398] Leaking SignalData structure 0x7b08003687c0 after lost signal to thread 431
W20260709 12:17:37.928207 32729 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0540 after lost signal to thread 505
W20260709 12:17:38.750094 22107 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 714)
W20260709 12:17:38.838347 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:39.274435   373 consensus_peers.cc:597] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.417128   447 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.635087   447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111) [suppressed 99 similar messages]
W20260709 12:17:39.652701   447 consensus_peers.cc:597] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.720283   373 consensus_peers.cc:597] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.747669   373 consensus_peers.cc:597] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.750626 22107 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 714)
W20260709 12:17:39.803020   373 consensus_peers.cc:597] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:39.839625 32746 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:17:39.875083   447 consensus_peers.cc:597] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:40.018682   447 consensus_peers.cc:597] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:17:40.076261   447 consensus_peers.cc:597] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:17:40.351549 22107 tablet_server.cc:196] TabletServer@127.21.150.193:44501 shutdown complete.
I20260709 12:17:40.416326 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:17:40.424787   445 consensus_peers.cc:597] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe -> Peer 7dde1735325f414bbdb50b608d80dca1 (127.21.150.194:40963): Couldn't send request to peer 7dde1735325f414bbdb50b608d80dca1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:40.459146 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:40.460103 22107 tablet_replica.cc:333] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.461078 22107 raft_consensus.cc:2243] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.462495 22107 raft_consensus.cc:2272] T a761e98905764ff5af3aa30229ca50d4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.465934 22107 tablet_replica.cc:333] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.466770 22107 raft_consensus.cc:2243] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.467940 22107 raft_consensus.cc:2272] T 58d0d9e0f0f949099570bddfa5106e8f P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.471135 22107 tablet_replica.cc:333] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.471890 22107 raft_consensus.cc:2243] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.472656 22107 raft_consensus.cc:2272] T f2a3bb67994c4a9e804975daa26f99f6 P 7dde1735325f414bbdb50b608d80dca1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.475668 22107 tablet_replica.cc:333] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.476224 22107 raft_consensus.cc:2243] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.477409 22107 raft_consensus.cc:2272] T 11ee1edff6824b32bb78ced08ffba7f5 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.480365 22107 tablet_replica.cc:333] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.481055 22107 raft_consensus.cc:2243] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.482242 22107 raft_consensus.cc:2272] T d249715915f64545ba53f74e84a294e4 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.485263 22107 tablet_replica.cc:333] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.485973 22107 raft_consensus.cc:2243] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.486703 22107 raft_consensus.cc:2272] T aaae3f0f5bc646a6aeb204604f7d99d3 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.489751 22107 tablet_replica.cc:333] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.490356 22107 raft_consensus.cc:2243] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.491034 22107 raft_consensus.cc:2272] T 84f4d76718f446c78e1feb938aefc903 P 7dde1735325f414bbdb50b608d80dca1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.493814 22107 tablet_replica.cc:333] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.494473 22107 raft_consensus.cc:2243] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.495105 22107 raft_consensus.cc:2272] T 75e138f0842544c9903c3e030689ccb7 P 7dde1735325f414bbdb50b608d80dca1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.498194 22107 tablet_replica.cc:333] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.498768 22107 raft_consensus.cc:2243] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.499480 22107 raft_consensus.cc:2272] T 56b6194d92c34c089bf0c6a8ecef5930 P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.502379 22107 tablet_replica.cc:333] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1: stopping tablet replica
I20260709 12:17:40.502959 22107 raft_consensus.cc:2243] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.503575 22107 raft_consensus.cc:2272] T 49b3b5891cbb48d19e8dead71573964b P 7dde1735325f414bbdb50b608d80dca1 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:40.514312   447 consensus_peers.cc:597] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe -> Peer efe9c93a145a490daba616e633c14a8b (127.21.150.193:44501): Couldn't send request to peer efe9c93a145a490daba616e633c14a8b. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:44501: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:17:40.565688 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
I20260709 12:17:40.606184 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:17:40.654081 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:40.654948 22107 tablet_replica.cc:333] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.655869 22107 raft_consensus.cc:2243] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.656489 22107 raft_consensus.cc:2272] T a761e98905764ff5af3aa30229ca50d4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.660780 22107 tablet_replica.cc:333] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.661935 22107 raft_consensus.cc:2243] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.662823 22107 raft_consensus.cc:2272] T 58d0d9e0f0f949099570bddfa5106e8f P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.665642 22107 tablet_replica.cc:333] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.666237 22107 raft_consensus.cc:2243] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.666752 22107 raft_consensus.cc:2272] T 11ee1edff6824b32bb78ced08ffba7f5 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.669255 22107 tablet_replica.cc:333] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.669884 22107 raft_consensus.cc:2243] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:40.670454 22107 raft_consensus.cc:2272] T d249715915f64545ba53f74e84a294e4 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.672967 22107 tablet_replica.cc:333] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.673568 22107 raft_consensus.cc:2243] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.674645 22107 raft_consensus.cc:2272] T 49b3b5891cbb48d19e8dead71573964b P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.677078 22107 tablet_replica.cc:333] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.677709 22107 raft_consensus.cc:2243] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.678785 22107 raft_consensus.cc:2272] T 56b6194d92c34c089bf0c6a8ecef5930 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.681115 22107 tablet_replica.cc:333] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.681749 22107 raft_consensus.cc:2243] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.682705 22107 raft_consensus.cc:2272] T 84f4d76718f446c78e1feb938aefc903 P 784b9febfff245dda6a27c4be73884fe [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.685184 22107 tablet_replica.cc:333] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.685787 22107 raft_consensus.cc:2243] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.686836 22107 raft_consensus.cc:2272] T aaae3f0f5bc646a6aeb204604f7d99d3 P 784b9febfff245dda6a27c4be73884fe [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.689347 22107 tablet_replica.cc:333] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.689831 22107 raft_consensus.cc:2243] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.690763 22107 raft_consensus.cc:2272] T 75e138f0842544c9903c3e030689ccb7 P 784b9febfff245dda6a27c4be73884fe [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.693069 22107 tablet_replica.cc:333] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe: stopping tablet replica
I20260709 12:17:40.693600 22107 raft_consensus.cc:2243] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.694562 22107 raft_consensus.cc:2272] T f2a3bb67994c4a9e804975daa26f99f6 P 784b9febfff245dda6a27c4be73884fe [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.731271 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:17:40.777199 22107 master.cc:562] Master@127.21.150.254:43431 shutting down...
I20260709 12:17:40.821739 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:40.822644 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:40.823073 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8bcd7c280ec14efa90134a5a1346a12b: stopping tablet replica
I20260709 12:17:41.613521 22107 master.cc:584] Master@127.21.150.254:43431 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (19975 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:17:41.662962 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:46853
I20260709 12:17:41.664239 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:41.670491   849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.670680   848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.671267   851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:41.672595 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:41.673947 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:41.674124 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:41.674309 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599461674287 us; error 0 us; skew 500 ppm
I20260709 12:17:41.674870 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:41.677346 22107 webserver.cc:533] Webserver started at http://127.21.150.254:45061/ using document root <none> and password file <none>
I20260709 12:17:41.677991 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:41.678187 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:41.678476 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:41.679680 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "58a00f7f6e1d438e85a443f0c7d45da0"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:41.684166 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:41.687647   856 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:41.688393 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:17:41.688700 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "58a00f7f6e1d438e85a443f0c7d45da0"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:41.689000 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:41.704756 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:41.706086 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:41.746554 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:46853
I20260709 12:17:41.746658   907 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:46853 every 8 connection(s)
I20260709 12:17:41.750597   908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:41.763541   908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0: Bootstrap starting.
I20260709 12:17:41.769089   908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:41.773391   908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0: No bootstrap required, opened a new log
I20260709 12:17:41.775712   908 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER }
I20260709 12:17:41.776150   908 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:41.776374   908 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a00f7f6e1d438e85a443f0c7d45da0, State: Initialized, Role: FOLLOWER
I20260709 12:17:41.777065   908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER }
I20260709 12:17:41.777551   908 raft_consensus.cc:399] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:41.777769   908 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:41.778067   908 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:41.783035   908 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER }
I20260709 12:17:41.783528   908 leader_election.cc:304] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58a00f7f6e1d438e85a443f0c7d45da0; no voters: 
I20260709 12:17:41.784617   908 leader_election.cc:290] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:41.784993   911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:41.786365   911 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 1 LEADER]: Becoming Leader. State: Replica: 58a00f7f6e1d438e85a443f0c7d45da0, State: Running, Role: LEADER
I20260709 12:17:41.787083   911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER }
I20260709 12:17:41.787606   908 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:41.793205   913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a00f7f6e1d438e85a443f0c7d45da0. Latest consensus state: current_term: 1 leader_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER } }
I20260709 12:17:41.793344   912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a00f7f6e1d438e85a443f0c7d45da0" member_type: VOTER } }
I20260709 12:17:41.793875   913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:41.794016   912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:41.795866   919 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:41.802064   919 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:41.803349 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:41.810917   919 catalog_manager.cc:1383] Generated new cluster ID: 7f9a25b5c4cf43cc9e1634629b682c85
I20260709 12:17:41.811188   919 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:41.827858   919 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:41.829142   919 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:41.844445   919 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0: Generated new TSK 0
I20260709 12:17:41.845093   919 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:41.870273 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:41.876432   929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.877583   930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.880965   932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:41.881578 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:41.883126 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:41.883313 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:41.883436 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599461883426 us; error 0 us; skew 500 ppm
I20260709 12:17:41.883921 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:41.886226 22107 webserver.cc:533] Webserver started at http://127.21.150.193:33867/ using document root <none> and password file <none>
I20260709 12:17:41.886662 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:41.886828 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:41.887038 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:41.888077 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "b2131bf3210c4c2a8945ba5154782e60"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:41.892774 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:41.896134   937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:41.896915 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:17:41.897230 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "b2131bf3210c4c2a8945ba5154782e60"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:41.897562 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:41.921811 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:41.923231 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:41.924983 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:41.927502 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:41.927727 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:41.927985 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:41.928174 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:41.969548 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:41429
I20260709 12:17:41.969652   999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:41429 every 8 connection(s)
I20260709 12:17:41.974817 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:41.983527  1004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.983706  1005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:41.986644  1007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:41.987474 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:41.988709 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:41.988983 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:41.989200 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599461989180 us; error 0 us; skew 500 ppm
I20260709 12:17:41.989975 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:41.991683  1000 heartbeater.cc:344] Connected to a master server at 127.21.150.254:46853
I20260709 12:17:41.992048  1000 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:41.992784  1000 heartbeater.cc:507] Master 127.21.150.254:46853 requested a full tablet report, sending...
I20260709 12:17:41.992805 22107 webserver.cc:533] Webserver started at http://127.21.150.194:44965/ using document root <none> and password file <none>
I20260709 12:17:41.993388 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:41.993853 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:41.994246 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:41.995421   873 ts_manager.cc:194] Registered new tserver with Master: b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429)
I20260709 12:17:41.995868 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:41.998052   873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52686
I20260709 12:17:42.001816 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:17:42.005208  1012 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.006080 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:17:42.006376 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
format_stamp: "Formatted at 2026-07-09 12:17:41 on dist-test-slave-zctp"
I20260709 12:17:42.006644 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:42.020156 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:42.021423 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:42.022984 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:42.025128 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:42.025362 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.025624 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:42.025796 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.067572 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:33827
I20260709 12:17:42.067675  1074 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:33827 every 8 connection(s)
I20260709 12:17:42.074131 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:42.081634  1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:42.082391  1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:42.087275 22107 server_base.cc:1061] running on GCE node
W20260709 12:17:42.087643  1081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:42.088536  1075 heartbeater.cc:344] Connected to a master server at 127.21.150.254:46853
I20260709 12:17:42.088845 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:17:42.088984  1075 heartbeater.cc:461] Registering TS with master...
W20260709 12:17:42.089171 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:42.089529 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599462089502 us; error 0 us; skew 500 ppm
I20260709 12:17:42.090085  1075 heartbeater.cc:507] Master 127.21.150.254:46853 requested a full tablet report, sending...
I20260709 12:17:42.090178 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:42.092356   873 ts_manager.cc:194] Registered new tserver with Master: 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.093035 22107 webserver.cc:533] Webserver started at http://127.21.150.195:36039/ using document root <none> and password file <none>
I20260709 12:17:42.093665 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:42.093881 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:42.094008   873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52698
I20260709 12:17:42.094172 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:42.095729 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "33d40b98209e4f238f14897f6c816c51"
format_stamp: "Formatted at 2026-07-09 12:17:42 on dist-test-slave-zctp"
I20260709 12:17:42.100440 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:42.103806  1086 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.104488 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:17:42.104780 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "33d40b98209e4f238f14897f6c816c51"
format_stamp: "Formatted at 2026-07-09 12:17:42 on dist-test-slave-zctp"
I20260709 12:17:42.105055 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:42.120874 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:42.122219 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:42.123771 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:42.126173 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:42.126406 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.126662 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:42.126843 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:42.167514 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:39189
I20260709 12:17:42.167596  1148 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:39189 every 8 connection(s)
I20260709 12:17:42.181815  1149 heartbeater.cc:344] Connected to a master server at 127.21.150.254:46853
I20260709 12:17:42.182164  1149 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:42.182870  1149 heartbeater.cc:507] Master 127.21.150.254:46853 requested a full tablet report, sending...
I20260709 12:17:42.184763   873 ts_manager.cc:194] Registered new tserver with Master: 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:42.185389 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014789948s
I20260709 12:17:42.186450   873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52706
I20260709 12:17:42.213627   873 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52716:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:42.215840   873 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:42.273119  1039 tablet_service.cc:1511] Processing CreateTablet for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.273276  1040 tablet_service.cc:1511] Processing CreateTablet for tablet 3373f8334f324b2f90906b7447b4e531 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.274710  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.275347  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3373f8334f324b2f90906b7447b4e531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.281282   965 tablet_service.cc:1511] Processing CreateTablet for tablet 3373f8334f324b2f90906b7447b4e531 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.282860   965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3373f8334f324b2f90906b7447b4e531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.285624   964 tablet_service.cc:1511] Processing CreateTablet for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.286643  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 3373f8334f324b2f90906b7447b4e531 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.287230   964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.288108  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3373f8334f324b2f90906b7447b4e531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.291697  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=2b59803900884f158e2cd253049bfe3a]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.293066  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4aebb2e4852e4a26a2eb0248b2b0b4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.302315  1164 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: Bootstrap starting.
I20260709 12:17:42.311435  1164 tablet_bootstrap.cc:654] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.311486  1165 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: Bootstrap starting.
I20260709 12:17:42.323956  1164 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: No bootstrap required, opened a new log
I20260709 12:17:42.324537  1164 ts_tablet_manager.cc:1403] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.007s
I20260709 12:17:42.325088  1165 tablet_bootstrap.cc:654] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.327672  1164 raft_consensus.cc:359] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.328650  1164 raft_consensus.cc:385] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.329003  1164 raft_consensus.cc:740] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e2e1c92fcb4ccfa3729f60429927a2, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.329939  1164 consensus_queue.cc:260] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.331748  1167 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: Bootstrap starting.
I20260709 12:17:42.333102  1165 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: No bootstrap required, opened a new log
I20260709 12:17:42.333374  1075 heartbeater.cc:499] Master 127.21.150.254:46853 was elected leader, sending a full tablet report...
I20260709 12:17:42.333719  1165 ts_tablet_manager.cc:1403] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 12:17:42.333976  1164 ts_tablet_manager.cc:1434] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:17:42.336951  1165 raft_consensus.cc:359] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.337839  1165 raft_consensus.cc:385] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.338229  1165 raft_consensus.cc:740] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.338932  1164 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: Bootstrap starting.
I20260709 12:17:42.339280  1167 tablet_bootstrap.cc:654] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.339246  1165 consensus_queue.cc:260] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.347955  1164 tablet_bootstrap.cc:654] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.350888  1165 ts_tablet_manager.cc:1434] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 12:17:42.352285  1000 heartbeater.cc:499] Master 127.21.150.254:46853 was elected leader, sending a full tablet report...
I20260709 12:17:42.352892  1165 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: Bootstrap starting.
I20260709 12:17:42.358736  1167 tablet_bootstrap.cc:492] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: No bootstrap required, opened a new log
I20260709 12:17:42.359341  1167 ts_tablet_manager.cc:1403] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: Time spent bootstrapping tablet: real 0.028s	user 0.008s	sys 0.013s
I20260709 12:17:42.361936  1165 tablet_bootstrap.cc:654] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.362568  1167 raft_consensus.cc:359] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.363358  1167 raft_consensus.cc:385] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.363660  1167 raft_consensus.cc:740] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.364449  1167 consensus_queue.cc:260] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.368781  1167 ts_tablet_manager.cc:1434] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:17:42.369035  1165 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: No bootstrap required, opened a new log
I20260709 12:17:42.369189  1164 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: No bootstrap required, opened a new log
I20260709 12:17:42.369563  1149 heartbeater.cc:499] Master 127.21.150.254:46853 was elected leader, sending a full tablet report...
I20260709 12:17:42.369678  1165 ts_tablet_manager.cc:1403] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 12:17:42.369822  1164 ts_tablet_manager.cc:1403] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.012s
I20260709 12:17:42.370630  1167 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: Bootstrap starting.
I20260709 12:17:42.373029  1165 raft_consensus.cc:359] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.373323  1164 raft_consensus.cc:359] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.374771  1164 raft_consensus.cc:385] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.375149  1164 raft_consensus.cc:740] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e2e1c92fcb4ccfa3729f60429927a2, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.373932  1165 raft_consensus.cc:385] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.376264  1165 raft_consensus.cc:740] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.376204  1164 consensus_queue.cc:260] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.377029  1167 tablet_bootstrap.cc:654] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.377279  1165 consensus_queue.cc:260] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.379727  1165 ts_tablet_manager.cc:1434] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:17:42.381752  1164 ts_tablet_manager.cc:1434] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:17:42.383541  1167 tablet_bootstrap.cc:492] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: No bootstrap required, opened a new log
I20260709 12:17:42.384160  1167 ts_tablet_manager.cc:1403] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.002s
I20260709 12:17:42.387198  1167 raft_consensus.cc:359] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.387776  1167 raft_consensus.cc:385] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.388034  1167 raft_consensus.cc:740] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.388808  1173 raft_consensus.cc:493] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.388787  1167 consensus_queue.cc:260] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.389475  1173 raft_consensus.cc:515] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.392386  1173 leader_election.cc:290] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.392565  1167 ts_tablet_manager.cc:1434] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:17:42.404299  1170 raft_consensus.cc:493] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.404940  1170 raft_consensus.cc:515] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.408859   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60" is_pre_election: true
I20260709 12:17:42.410094   975 raft_consensus.cc:2468] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.412775  1170 leader_election.cc:290] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:42.414011  1169 raft_consensus.cc:493] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.414577  1169 raft_consensus.cc:515] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.417382  1169 leader_election.cc:290] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:42.423890  1088 leader_election.cc:304] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.425401  1176 raft_consensus.cc:2804] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:42.425971  1176 raft_consensus.cc:493] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.426591  1176 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.434458  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" is_pre_election: true
I20260709 12:17:42.435320  1050 raft_consensus.cc:2468] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.438664  1176 raft_consensus.cc:515] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.443235   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
I20260709 12:17:42.443871   975 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.447119  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d40b98209e4f238f14897f6c816c51" is_pre_election: true
I20260709 12:17:42.447350  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d40b98209e4f238f14897f6c816c51" is_pre_election: true
I20260709 12:17:42.447746  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:42.447937  1124 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 0.
I20260709 12:17:42.448303  1123 raft_consensus.cc:2393] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 92e2e1c92fcb4ccfa3729f60429927a2 in current term 1: Already voted for candidate 33d40b98209e4f238f14897f6c816c51 in this term.
I20260709 12:17:42.448618  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" is_pre_election: true
I20260709 12:17:42.448374  1050 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.449290   939 leader_election.cc:304] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.450533  1170 raft_consensus.cc:2804] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:42.450896  1170 raft_consensus.cc:493] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.451206  1170 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.449319  1049 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 0.
I20260709 12:17:42.453697   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60" is_pre_election: true
I20260709 12:17:42.454404   975 raft_consensus.cc:2468] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.455693  1088 leader_election.cc:304] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.456429  1173 raft_consensus.cc:2804] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:42.443235  1176 leader_election.cc:290] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Requested vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.458931  1014 leader_election.cc:304] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 92e2e1c92fcb4ccfa3729f60429927a2; no voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60
I20260709 12:17:42.459725  1170 raft_consensus.cc:515] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.460894  1050 raft_consensus.cc:2468] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.461422  1169 raft_consensus.cc:2749] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:42.462586  1170 leader_election.cc:290] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 1 election: Requested vote from peers 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:42.462886  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:42.463564  1049 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.463760  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33d40b98209e4f238f14897f6c816c51"
I20260709 12:17:42.464542  1123 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.465478  1173 raft_consensus.cc:697] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Leader. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:42.466548  1173 consensus_queue.cc:237] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.471901  1123 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 1.
I20260709 12:17:42.472661  1049 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 1.
I20260709 12:17:42.473312   939 leader_election.cc:304] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.474902  1170 raft_consensus.cc:2804] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:42.476245  1170 raft_consensus.cc:697] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 LEADER]: Becoming Leader. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:42.477274  1170 consensus_queue.cc:237] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.477948   871 catalog_manager.cc:5697] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d40b98209e4f238f14897f6c816c51 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "33d40b98209e4f238f14897f6c816c51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:42.486235   873 catalog_manager.cc:5697] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193). New cstate: current_term: 1 leader_uuid: "b2131bf3210c4c2a8945ba5154782e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:42.553071   873 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52726:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:42.555092   873 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:42.581466   965 tablet_service.cc:1511] Processing CreateTablet for tablet 240aa5b7c0374f179cec46b01569fd85 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.582870   965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240aa5b7c0374f179cec46b01569fd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.582794  1039 tablet_service.cc:1511] Processing CreateTablet for tablet 240aa5b7c0374f179cec46b01569fd85 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.584257  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240aa5b7c0374f179cec46b01569fd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.584614  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 240aa5b7c0374f179cec46b01569fd85 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:17:42.586042  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 240aa5b7c0374f179cec46b01569fd85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.587742  1040 tablet_service.cc:1511] Processing CreateTablet for tablet da2e1efd892a415e990ec6ced0c34cf6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.589141  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da2e1efd892a415e990ec6ced0c34cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.593169   964 tablet_service.cc:1511] Processing CreateTablet for tablet da2e1efd892a415e990ec6ced0c34cf6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.594498   964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da2e1efd892a415e990ec6ced0c34cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.601205  1165 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: Bootstrap starting.
I20260709 12:17:42.604882  1113 tablet_service.cc:1511] Processing CreateTablet for tablet da2e1efd892a415e990ec6ced0c34cf6 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=00a2d98a879848d588a2aa08d83554c1]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:17:42.606437  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da2e1efd892a415e990ec6ced0c34cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:42.607609  1164 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: Bootstrap starting.
I20260709 12:17:42.608596  1165 tablet_bootstrap.cc:654] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.614980  1164 tablet_bootstrap.cc:654] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.624577  1164 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: No bootstrap required, opened a new log
I20260709 12:17:42.625155  1164 ts_tablet_manager.cc:1403] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.000s
I20260709 12:17:42.627060  1167 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: Bootstrap starting.
I20260709 12:17:42.628825  1164 raft_consensus.cc:359] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.629496  1165 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: No bootstrap required, opened a new log
I20260709 12:17:42.629681  1164 raft_consensus.cc:385] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.630028  1164 raft_consensus.cc:740] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e2e1c92fcb4ccfa3729f60429927a2, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.630043  1165 ts_tablet_manager.cc:1403] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 12:17:42.631122  1164 consensus_queue.cc:260] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.633137  1164 ts_tablet_manager.cc:1434] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:42.634246  1164 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Bootstrap starting.
I20260709 12:17:42.634620  1167 tablet_bootstrap.cc:654] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.635383  1165 raft_consensus.cc:359] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.636202  1165 raft_consensus.cc:385] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.636538  1165 raft_consensus.cc:740] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.637362  1165 consensus_queue.cc:260] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.639746  1165 ts_tablet_manager.cc:1434] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:17:42.640864  1165 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: Bootstrap starting.
I20260709 12:17:42.644654  1164 tablet_bootstrap.cc:654] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.646842  1167 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: No bootstrap required, opened a new log
I20260709 12:17:42.647511  1167 ts_tablet_manager.cc:1403] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.008s
I20260709 12:17:42.647876  1165 tablet_bootstrap.cc:654] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.650786  1167 raft_consensus.cc:359] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.651795  1167 raft_consensus.cc:385] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.652161  1167 raft_consensus.cc:740] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.653129  1167 consensus_queue.cc:260] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.655483  1167 ts_tablet_manager.cc:1434] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:42.656661  1167 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: Bootstrap starting.
I20260709 12:17:42.663412  1167 tablet_bootstrap.cc:654] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:42.666437  1165 tablet_bootstrap.cc:492] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: No bootstrap required, opened a new log
I20260709 12:17:42.667116  1165 ts_tablet_manager.cc:1403] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.018s
I20260709 12:17:42.670446  1165 raft_consensus.cc:359] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.671243  1165 raft_consensus.cc:385] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.671582  1165 raft_consensus.cc:740] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.672394  1165 consensus_queue.cc:260] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.674510  1165 ts_tablet_manager.cc:1434] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:42.677323  1164 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: No bootstrap required, opened a new log
I20260709 12:17:42.677644  1167 tablet_bootstrap.cc:492] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: No bootstrap required, opened a new log
I20260709 12:17:42.678081  1164 ts_tablet_manager.cc:1403] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent bootstrapping tablet: real 0.044s	user 0.021s	sys 0.015s
I20260709 12:17:42.678233  1167 ts_tablet_manager.cc:1403] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:17:42.681329  1164 raft_consensus.cc:359] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.681666  1167 raft_consensus.cc:359] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.682246  1164 raft_consensus.cc:385] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.682516  1167 raft_consensus.cc:385] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:42.682646  1164 raft_consensus.cc:740] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e2e1c92fcb4ccfa3729f60429927a2, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.682921  1167 raft_consensus.cc:740] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Initialized, Role: FOLLOWER
I20260709 12:17:42.683667  1164 consensus_queue.cc:260] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.683895  1167 consensus_queue.cc:260] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.686028  1164 ts_tablet_manager.cc:1434] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:17:42.686370  1167 ts_tablet_manager.cc:1434] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:42.687793  1173 raft_consensus.cc:493] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.688336  1173 raft_consensus.cc:515] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.690464  1173 leader_election.cc:290] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.691355   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60" is_pre_election: true
I20260709 12:17:42.691708  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" is_pre_election: true
I20260709 12:17:42.692126   975 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.692420  1049 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.693490  1088 leader_election.cc:304] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.694382  1173 raft_consensus.cc:2804] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:42.694751  1173 raft_consensus.cc:493] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.695123  1173 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.702626  1173 raft_consensus.cc:515] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.704180  1173 leader_election.cc:290] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Requested vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.705080   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
I20260709 12:17:42.705317  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:42.705701   975 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.705955  1049 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.711424  1049 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.711967   975 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.712718  1090 leader_election.cc:304] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, 92e2e1c92fcb4ccfa3729f60429927a2; no voters: 
I20260709 12:17:42.713644  1173 raft_consensus.cc:2804] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:42.714056  1173 raft_consensus.cc:697] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Leader. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:42.714891  1173 consensus_queue.cc:237] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.721589   873 catalog_manager.cc:5697] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d40b98209e4f238f14897f6c816c51 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "33d40b98209e4f238f14897f6c816c51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: HEALTHY } } }
W20260709 12:17:42.799103   921 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:42.817608  1173 raft_consensus.cc:493] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:42.818222  1173 raft_consensus.cc:515] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.819973  1173 leader_election.cc:290] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.820919   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60" is_pre_election: true
I20260709 12:17:42.821061  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" is_pre_election: true
I20260709 12:17:42.821620   975 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.821756  1049 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 0.
I20260709 12:17:42.822633  1088 leader_election.cc:304] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.823372  1173 raft_consensus.cc:2804] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:42.823757  1173 raft_consensus.cc:493] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:42.824065  1173 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.829744  1173 raft_consensus.cc:515] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.831563  1173 leader_election.cc:290] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Requested vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:42.832341   975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
I20260709 12:17:42.832568  1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:42.832898   975 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.833149  1049 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:42.837836   975 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.837950  1049 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 1.
I20260709 12:17:42.838956  1088 leader_election.cc:304] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:42.839632  1173 raft_consensus.cc:2804] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:42.840114  1173 raft_consensus.cc:697] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Leader. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:42.840754  1173 consensus_queue.cc:237] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.848093   873 catalog_manager.cc:5697] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33d40b98209e4f238f14897f6c816c51 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "33d40b98209e4f238f14897f6c816c51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:42.891992  1123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531"
dest_uuid: "33d40b98209e4f238f14897f6c816c51"
mode: GRACEFUL
new_leader_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56572
I20260709 12:17:42.892598  1123 raft_consensus.cc:606] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Received request to transfer leadership to TS b2131bf3210c4c2a8945ba5154782e60
I20260709 12:17:42.905998   975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6"
dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
mode: GRACEFUL
new_leader_uuid: "33d40b98209e4f238f14897f6c816c51"
 from {username='slave'} at 127.0.0.1:56942
I20260709 12:17:42.906594   975 raft_consensus.cc:606] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 LEADER]: Received request to transfer leadership to TS 33d40b98209e4f238f14897f6c816c51
I20260709 12:17:42.912433  1123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85"
dest_uuid: "33d40b98209e4f238f14897f6c816c51"
mode: GRACEFUL
new_leader_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56572
I20260709 12:17:42.912976  1123 raft_consensus.cc:606] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Received request to transfer leadership to TS b2131bf3210c4c2a8945ba5154782e60
I20260709 12:17:42.918238  1123 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6"
dest_uuid: "33d40b98209e4f238f14897f6c816c51"
mode: GRACEFUL
new_leader_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56572
I20260709 12:17:42.918785  1123 raft_consensus.cc:606] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Received request to transfer leadership to TS b2131bf3210c4c2a8945ba5154782e60
I20260709 12:17:42.963418  1173 consensus_queue.cc:1048] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:42.971251  1183 raft_consensus.cc:993] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: : Instructing follower b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:42.972111  1173 raft_consensus.cc:1081] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Signalling peer b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:42.974900   974 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531"
dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56928
I20260709 12:17:42.977159   974 raft_consensus.cc:493] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:42.978216   974 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:42.989215   974 raft_consensus.cc:515] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:42.992816  1173 consensus_queue.cc:1048] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:42.995874  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:42.996057   974 leader_election.cc:290] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Requested vote from peers 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:42.997212  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3373f8334f324b2f90906b7447b4e531" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d40b98209e4f238f14897f6c816c51"
I20260709 12:17:42.998024  1123 raft_consensus.cc:3055] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:42.998416  1123 raft_consensus.cc:740] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:42.999541  1123 consensus_queue.cc:260] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.001077  1123 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.005793  1170 consensus_queue.cc:1048] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:43.013549  1123 raft_consensus.cc:2468] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 2.
I20260709 12:17:43.014959   939 leader_election.cc:304] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, b2131bf3210c4c2a8945ba5154782e60; no voters: 92e2e1c92fcb4ccfa3729f60429927a2
I20260709 12:17:43.016192  1170 raft_consensus.cc:2804] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.019363  1170 raft_consensus.cc:697] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Becoming Leader. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:43.020102  1170 consensus_queue.cc:237] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.027599   869 catalog_manager.cc:5697] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 reported cstate change: term changed from 1 to 2, leader changed from 33d40b98209e4f238f14897f6c816c51 (127.21.150.195) to b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "b2131bf3210c4c2a8945ba5154782e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.040277  1209 consensus_queue.cc:1048] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:43.050467  1210 raft_consensus.cc:993] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: : Instructing follower 33d40b98209e4f238f14897f6c816c51 to start an election
I20260709 12:17:43.051085  1209 raft_consensus.cc:1081] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 LEADER]: Signalling peer 33d40b98209e4f238f14897f6c816c51 to start an election
I20260709 12:17:43.052778  1123 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6"
dest_uuid: "33d40b98209e4f238f14897f6c816c51"
 from {username='slave'} at 127.0.0.1:56560
I20260709 12:17:43.053488  1123 raft_consensus.cc:493] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:43.053903  1123 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.060534  1123 raft_consensus.cc:515] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.063016  1123 leader_election.cc:290] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 2 election: Requested vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827)
I20260709 12:17:43.063710   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
I20260709 12:17:43.064395   974 raft_consensus.cc:3055] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:43.064738   974 raft_consensus.cc:740] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:43.064965  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4aebb2e4852e4a26a2eb0248b2b0b4e6" candidate_uuid: "33d40b98209e4f238f14897f6c816c51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:43.065583  1050 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.065644   974 consensus_queue.cc:260] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.067067   974 raft_consensus.cc:3060] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.073093  1050 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 2.
I20260709 12:17:43.074318  1090 leader_election.cc:304] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, 92e2e1c92fcb4ccfa3729f60429927a2; no voters: 
I20260709 12:17:43.074741   974 raft_consensus.cc:2468] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d40b98209e4f238f14897f6c816c51 in term 2.
I20260709 12:17:43.075248  1173 raft_consensus.cc:2804] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.075752  1173 raft_consensus.cc:697] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 2 LEADER]: Becoming Leader. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:43.076592  1173 consensus_queue.cc:237] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.082909   873 catalog_manager.cc:5697] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 reported cstate change: term changed from 1 to 2, leader changed from b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193) to 33d40b98209e4f238f14897f6c816c51 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "33d40b98209e4f238f14897f6c816c51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:43.210786  1173 consensus_queue.cc:1048] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:43.221506  1202 consensus_queue.cc:1048] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:43.233369  1183 raft_consensus.cc:993] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: : Instructing follower b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:43.233881  1214 raft_consensus.cc:1081] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Signalling peer b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:43.235608   974 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6"
dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56928
I20260709 12:17:43.236117   974 raft_consensus.cc:493] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:43.236439   974 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.243212   974 raft_consensus.cc:515] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.245685   974 leader_election.cc:290] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Requested vote from peers 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:43.246555  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d40b98209e4f238f14897f6c816c51"
I20260709 12:17:43.247218  1123 raft_consensus.cc:3055] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:43.247452  1123 raft_consensus.cc:740] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:43.248227  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da2e1efd892a415e990ec6ced0c34cf6" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:43.248224  1123 consensus_queue.cc:260] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.249017  1050 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.249912  1123 raft_consensus.cc:3060] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.255350  1050 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 2.
I20260709 12:17:43.255858  1123 raft_consensus.cc:2468] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 2.
I20260709 12:17:43.256534   941 leader_election.cc:304] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92e2e1c92fcb4ccfa3729f60429927a2, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:43.257444  1209 raft_consensus.cc:2804] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.258088  1209 raft_consensus.cc:697] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Becoming Leader. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:43.258828  1209 consensus_queue.cc:237] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.264848   869 catalog_manager.cc:5697] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 reported cstate change: term changed from 1 to 2, leader changed from 33d40b98209e4f238f14897f6c816c51 (127.21.150.195) to b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "b2131bf3210c4c2a8945ba5154782e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.297313  1214 consensus_queue.cc:1048] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:43.309082  1214 consensus_queue.cc:1048] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:43.317142  1202 raft_consensus.cc:993] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: : Instructing follower b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:43.317620  1214 raft_consensus.cc:1081] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Signalling peer b2131bf3210c4c2a8945ba5154782e60 to start an election
I20260709 12:17:43.319051   974 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85"
dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
 from {username='slave'} at 127.0.0.1:56928
I20260709 12:17:43.319569   974 raft_consensus.cc:493] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:43.319880   974 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.325282   974 raft_consensus.cc:515] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.327064   974 leader_election.cc:290] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Requested vote from peers 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194:33827), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:43.328123  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
I20260709 12:17:43.328258  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "b2131bf3210c4c2a8945ba5154782e60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "33d40b98209e4f238f14897f6c816c51"
I20260709 12:17:43.328809  1050 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.328904  1123 raft_consensus.cc:3055] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:43.329337  1123 raft_consensus.cc:740] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 33d40b98209e4f238f14897f6c816c51, State: Running, Role: LEADER
I20260709 12:17:43.330168  1123 consensus_queue.cc:260] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.331451  1123 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.334749  1050 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 2.
I20260709 12:17:43.335836   941 leader_election.cc:304] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92e2e1c92fcb4ccfa3729f60429927a2, b2131bf3210c4c2a8945ba5154782e60; no voters: 
I20260709 12:17:43.336517  1209 raft_consensus.cc:2804] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:43.336920  1209 raft_consensus.cc:697] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Becoming Leader. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:43.337031  1123 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2131bf3210c4c2a8945ba5154782e60 in term 2.
I20260709 12:17:43.337770  1209 consensus_queue.cc:237] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.343662   873 catalog_manager.cc:5697] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 reported cstate change: term changed from 1 to 2, leader changed from 33d40b98209e4f238f14897f6c816c51 (127.21.150.195) to b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193). New cstate: current_term: 2 leader_uuid: "b2131bf3210c4c2a8945ba5154782e60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.403774 22107 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:17:43.405049  1050 raft_consensus.cc:3060] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:43.406049 22107 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:17:43.406611 22107 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:17:43.412873  1050 raft_consensus.cc:1275] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.414465  1209 consensus_queue.cc:1048] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.421078   974 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85"
dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
mode: GRACEFUL
new_leader_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
 from {username='slave'} at 127.0.0.1:56956
I20260709 12:17:43.421747   974 raft_consensus.cc:606] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Received request to transfer leadership to TS 92e2e1c92fcb4ccfa3729f60429927a2
I20260709 12:17:43.424937  1123 raft_consensus.cc:1275] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.426014 22107 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:17:43.426623  1170 consensus_queue.cc:1048] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.601236   974 raft_consensus.cc:1275] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Refusing update from remote peer 33d40b98209e4f238f14897f6c816c51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.602734  1214 consensus_queue.cc:1048] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.611331  1050 raft_consensus.cc:1275] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Refusing update from remote peer 33d40b98209e4f238f14897f6c816c51: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.612998  1173 consensus_queue.cc:1048] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.719019  1050 raft_consensus.cc:1275] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.720397  1170 consensus_queue.cc:1048] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.725307  1211 raft_consensus.cc:993] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: : Instructing follower 92e2e1c92fcb4ccfa3729f60429927a2 to start an election
I20260709 12:17:43.725883  1210 raft_consensus.cc:1081] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Signalling peer 92e2e1c92fcb4ccfa3729f60429927a2 to start an election
I20260709 12:17:43.727859  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85"
dest_uuid: "92e2e1c92fcb4ccfa3729f60429927a2"
 from {username='slave'} at 127.0.0.1:37798
I20260709 12:17:43.728852  1049 raft_consensus.cc:493] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:43.729656  1049 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.730032  1123 raft_consensus.cc:1275] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.731631  1210 consensus_queue.cc:1048] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.739714  1049 raft_consensus.cc:515] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.742079  1049 leader_election.cc:290] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [CANDIDATE]: Term 3 election: Requested vote from peers b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193:41429), 33d40b98209e4f238f14897f6c816c51 (127.21.150.195:39189)
I20260709 12:17:43.743564   974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b2131bf3210c4c2a8945ba5154782e60"
I20260709 12:17:43.744266   974 raft_consensus.cc:3055] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:43.744305  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "240aa5b7c0374f179cec46b01569fd85" candidate_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "33d40b98209e4f238f14897f6c816c51"
I20260709 12:17:43.744879   974 raft_consensus.cc:740] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b2131bf3210c4c2a8945ba5154782e60, State: Running, Role: LEADER
I20260709 12:17:43.745215  1123 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.746134   974 consensus_queue.cc:260] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.747555   974 raft_consensus.cc:3060] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:43.753989  1123 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e2e1c92fcb4ccfa3729f60429927a2 in term 3.
I20260709 12:17:43.754992   974 raft_consensus.cc:2468] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e2e1c92fcb4ccfa3729f60429927a2 in term 3.
I20260709 12:17:43.755367  1014 leader_election.cc:304] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d40b98209e4f238f14897f6c816c51, 92e2e1c92fcb4ccfa3729f60429927a2; no voters: 
I20260709 12:17:43.756351  1207 raft_consensus.cc:2804] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:43.757324  1207 raft_consensus.cc:697] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 LEADER]: Becoming Leader. State: Replica: 92e2e1c92fcb4ccfa3729f60429927a2, State: Running, Role: LEADER
I20260709 12:17:43.758074  1207 consensus_queue.cc:237] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } }
I20260709 12:17:43.764192   871 catalog_manager.cc:5697] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 reported cstate change: term changed from 2 to 3, leader changed from b2131bf3210c4c2a8945ba5154782e60 (127.21.150.193) to 92e2e1c92fcb4ccfa3729f60429927a2 (127.21.150.194). New cstate: current_term: 3 leader_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2131bf3210c4c2a8945ba5154782e60" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:43.774606  1049 raft_consensus.cc:1275] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.776185  1210 consensus_queue.cc:1048] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e2e1c92fcb4ccfa3729f60429927a2" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 33827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:43.780654 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
I20260709 12:17:43.791777  1123 raft_consensus.cc:1275] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Refusing update from remote peer b2131bf3210c4c2a8945ba5154782e60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:43.793344  1209 consensus_queue.cc:1048] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33d40b98209e4f238f14897f6c816c51" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:43.825498 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:43.826282 22107 tablet_replica.cc:333] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60: stopping tablet replica
I20260709 12:17:43.826877 22107 raft_consensus.cc:2243] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:43.827751 22107 raft_consensus.cc:2272] T da2e1efd892a415e990ec6ced0c34cf6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.830183 22107 tablet_replica.cc:333] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60: stopping tablet replica
I20260709 12:17:43.830715 22107 raft_consensus.cc:2243] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.831144 22107 raft_consensus.cc:2272] T 240aa5b7c0374f179cec46b01569fd85 P b2131bf3210c4c2a8945ba5154782e60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.833782 22107 tablet_replica.cc:333] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60: stopping tablet replica
I20260709 12:17:43.834275 22107 raft_consensus.cc:2243] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.834658 22107 raft_consensus.cc:2272] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.836676 22107 tablet_replica.cc:333] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60: stopping tablet replica
I20260709 12:17:43.837103 22107 raft_consensus.cc:2243] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:43.837924 22107 raft_consensus.cc:2272] T 3373f8334f324b2f90906b7447b4e531 P b2131bf3210c4c2a8945ba5154782e60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.852161 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
I20260709 12:17:43.864530 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
I20260709 12:17:43.882411 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:43.883066 22107 tablet_replica.cc:333] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: stopping tablet replica
I20260709 12:17:43.883634 22107 raft_consensus.cc:2243] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:43.884301 22107 pending_rounds.cc:70] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Trying to abort 1 pending ops.
I20260709 12:17:43.884507 22107 pending_rounds.cc:77] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305623403560345600 op_type: NO_OP noop_request { }
I20260709 12:17:43.884819 22107 raft_consensus.cc:2889] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:17:43.885108 22107 raft_consensus.cc:2272] T 240aa5b7c0374f179cec46b01569fd85 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.887848 22107 tablet_replica.cc:333] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2: stopping tablet replica
I20260709 12:17:43.888804 22107 raft_consensus.cc:2243] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.889526 22107 raft_consensus.cc:2272] T da2e1efd892a415e990ec6ced0c34cf6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.891836 22107 tablet_replica.cc:333] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2: stopping tablet replica
I20260709 12:17:43.892386 22107 raft_consensus.cc:2243] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.892838 22107 raft_consensus.cc:2272] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.895218 22107 tablet_replica.cc:333] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2: stopping tablet replica
I20260709 12:17:43.895716 22107 raft_consensus.cc:2243] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.896142 22107 raft_consensus.cc:2272] T 3373f8334f324b2f90906b7447b4e531 P 92e2e1c92fcb4ccfa3729f60429927a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.918960 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
I20260709 12:17:43.932482 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:17:43.951545 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:43.952517 22107 tablet_replica.cc:333] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51: stopping tablet replica
I20260709 12:17:43.953210 22107 raft_consensus.cc:2243] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.953899 22107 raft_consensus.cc:2272] T 240aa5b7c0374f179cec46b01569fd85 P 33d40b98209e4f238f14897f6c816c51 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.956892 22107 tablet_replica.cc:333] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51: stopping tablet replica
I20260709 12:17:43.957523 22107 raft_consensus.cc:2243] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.957979 22107 raft_consensus.cc:2272] T da2e1efd892a415e990ec6ced0c34cf6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.960201 22107 tablet_replica.cc:333] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51: stopping tablet replica
I20260709 12:17:43.960680 22107 raft_consensus.cc:2243] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:43.961539 22107 raft_consensus.cc:2272] T 4aebb2e4852e4a26a2eb0248b2b0b4e6 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.963788 22107 tablet_replica.cc:333] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51: stopping tablet replica
I20260709 12:17:43.964247 22107 raft_consensus.cc:2243] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:43.964630 22107 raft_consensus.cc:2272] T 3373f8334f324b2f90906b7447b4e531 P 33d40b98209e4f238f14897f6c816c51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:43.988862 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:17:44.002159 22107 master.cc:562] Master@127.21.150.254:46853 shutting down...
I20260709 12:17:44.019496 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:44.020020 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:44.020289 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 58a00f7f6e1d438e85a443f0c7d45da0: stopping tablet replica
I20260709 12:17:44.039634 22107 master.cc:584] Master@127.21.150.254:46853 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2408 ms)
[----------] 6 tests from LeaderRebalancerTest (359559 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:17:44.071115 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:37815
I20260709 12:17:44.072345 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.077929  1221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.079656  1222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.081523  1224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.081573 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:44.082573 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.082799 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.082955 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599464082939 us; error 0 us; skew 500 ppm
I20260709 12:17:44.083488 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.086040 22107 webserver.cc:533] Webserver started at http://127.21.150.254:38383/ using document root <none> and password file <none>
I20260709 12:17:44.086553 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.086768 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.087046 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.088162 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "6af1e666f9854b0b8733b14e86ab5b11"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.092739 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:17:44.096364  1229 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.097246 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:44.097565 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "6af1e666f9854b0b8733b14e86ab5b11"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.097831 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.108817 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.109956 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.147668 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:37815
I20260709 12:17:44.147843  1280 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:37815 every 8 connection(s)
I20260709 12:17:44.151854  1281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.163354  1281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: Bootstrap starting.
I20260709 12:17:44.168031  1281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.172657  1281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: No bootstrap required, opened a new log
I20260709 12:17:44.174896  1281 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER }
I20260709 12:17:44.175284  1281 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.175640  1281 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af1e666f9854b0b8733b14e86ab5b11, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.176365  1281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER }
I20260709 12:17:44.176807  1281 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:44.177001  1281 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:44.177211  1281 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:44.182200  1281 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER }
I20260709 12:17:44.182663  1281 leader_election.cc:304] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af1e666f9854b0b8733b14e86ab5b11; no voters: 
I20260709 12:17:44.183734  1281 leader_election.cc:290] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:44.184091  1284 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:44.185509  1284 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 1 LEADER]: Becoming Leader. State: Replica: 6af1e666f9854b0b8733b14e86ab5b11, State: Running, Role: LEADER
I20260709 12:17:44.186180  1284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER }
I20260709 12:17:44.186656  1281 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:44.189159  1285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6af1e666f9854b0b8733b14e86ab5b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER } }
I20260709 12:17:44.189252  1286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af1e666f9854b0b8733b14e86ab5b11. Latest consensus state: current_term: 1 leader_uuid: "6af1e666f9854b0b8733b14e86ab5b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af1e666f9854b0b8733b14e86ab5b11" member_type: VOTER } }
I20260709 12:17:44.190074  1285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:44.190441  1286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:44.193222  1290 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:44.197890  1290 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:44.202401 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:17:44.206879  1290 catalog_manager.cc:1383] Generated new cluster ID: c343d69364f242c09cb5f5f463e05d5a
I20260709 12:17:44.207158  1290 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:44.229529  1290 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:44.230885  1290 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:44.250973  1290 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: Generated new TSK 0
I20260709 12:17:44.251653  1290 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:44.268911 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.275760  1302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.276876  1303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.278721  1305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.278771 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:44.279950 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.280185 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.280368 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599464280345 us; error 0 us; skew 500 ppm
I20260709 12:17:44.280900 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.284119 22107 webserver.cc:533] Webserver started at http://127.21.150.193:44289/ using document root <none> and password file <none>
I20260709 12:17:44.284772 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.285001 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.285285 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.286527 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "d972080851fd4e7784753234934d65bc"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.291505 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:17:44.295225  1310 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.296169 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:17:44.296561 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "d972080851fd4e7784753234934d65bc"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.296964 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.306169 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.307339 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.308904 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.311276 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.311514 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.311769 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.311950 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.356189 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:45907
I20260709 12:17:44.356312  1372 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:45907 every 8 connection(s)
I20260709 12:17:44.361455 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.369032  1377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.370082  1378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.374152 22107 server_base.cc:1061] running on GCE node
W20260709 12:17:44.375272  1380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.375952  1373 heartbeater.cc:344] Connected to a master server at 127.21.150.254:37815
I20260709 12:17:44.376346  1373 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:44.377233  1373 heartbeater.cc:507] Master 127.21.150.254:37815 requested a full tablet report, sending...
I20260709 12:17:44.377879 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.378154 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.378363 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599464378339 us; error 0 us; skew 500 ppm
I20260709 12:17:44.379130 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.379631  1246 ts_manager.cc:194] Registered new tserver with Master: d972080851fd4e7784753234934d65bc (127.21.150.193:45907)
I20260709 12:17:44.381783  1246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60740
I20260709 12:17:44.381884 22107 webserver.cc:533] Webserver started at http://127.21.150.194:37817/ using document root <none> and password file <none>
I20260709 12:17:44.382490 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.382721 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.383031 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.384140 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.389011 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:17:44.392557  1385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.393311 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:17:44.393687 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.394004 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.415125 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.416258 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.417811 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.420149 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.420329 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.420643 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.420789 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.463088 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:36391
I20260709 12:17:44.463214  1447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:36391 every 8 connection(s)
I20260709 12:17:44.468568 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:44.476166  1451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.476576  1452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:44.479036  1454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:44.480150 22107 server_base.cc:1061] running on GCE node
I20260709 12:17:44.481101 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:44.481389 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:44.481444  1448 heartbeater.cc:344] Connected to a master server at 127.21.150.254:37815
I20260709 12:17:44.481660 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599464481638 us; error 0 us; skew 500 ppm
I20260709 12:17:44.481962  1448 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:44.482429 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:44.482812  1448 heartbeater.cc:507] Master 127.21.150.254:37815 requested a full tablet report, sending...
I20260709 12:17:44.485414  1246 ts_manager.cc:194] Registered new tserver with Master: f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:44.485498 22107 webserver.cc:533] Webserver started at http://127.21.150.195:46825/ using document root <none> and password file <none>
I20260709 12:17:44.486258 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:44.486526 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:44.486828 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:44.487162  1246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60742
I20260709 12:17:44.488757 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.494777 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:44.498750  1459 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.499969 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:17:44.500259 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
format_stamp: "Formatted at 2026-07-09 12:17:44 on dist-test-slave-zctp"
I20260709 12:17:44.500530 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:44.513375 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:44.514556 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:44.516001 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:44.518455 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:44.518668 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.518959 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:44.519102 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:44.562919 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:39251
I20260709 12:17:44.563212  1521 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:39251 every 8 connection(s)
I20260709 12:17:44.580195  1522 heartbeater.cc:344] Connected to a master server at 127.21.150.254:37815
I20260709 12:17:44.580744  1522 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:44.581535  1522 heartbeater.cc:507] Master 127.21.150.254:37815 requested a full tablet report, sending...
I20260709 12:17:44.584057  1246 ts_manager.cc:194] Registered new tserver with Master: fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:44.584263 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017924329s
I20260709 12:17:44.585904  1246 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60746
I20260709 12:17:44.612102  1246 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60762:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:44.615061  1246 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:44.687212  1487 tablet_service.cc:1511] Processing CreateTablet for tablet fdeefaeba5c24b7a96d2413978ccc4e5 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:44.688720  1487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdeefaeba5c24b7a96d2413978ccc4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.688304  1486 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1a51e5426f4076a4f6ecc9887779c3 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:44.689894  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1a51e5426f4076a4f6ecc9887779c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.693956  1483 tablet_service.cc:1511] Processing CreateTablet for tablet bbd0f4c451664f72a1f75274abfb2496 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:44.694486  1482 tablet_service.cc:1511] Processing CreateTablet for tablet edc1782517cd4d2cbe842d3ea68e364c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:44.695583  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd0f4c451664f72a1f75274abfb2496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.696846  1482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc1782517cd4d2cbe842d3ea68e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.697778  1484 tablet_service.cc:1511] Processing CreateTablet for tablet 577caaca97e64a5caa06f4d2d500db4c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:44.699172  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577caaca97e64a5caa06f4d2d500db4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.700610  1481 tablet_service.cc:1511] Processing CreateTablet for tablet 97ddf31261234c61853d015f42a162e9 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:44.701992  1481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ddf31261234c61853d015f42a162e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.702546  1485 tablet_service.cc:1511] Processing CreateTablet for tablet ade1be5869844d02a7e6cf1f195dc559 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:44.703900  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1be5869844d02a7e6cf1f195dc559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.706955  1480 tablet_service.cc:1511] Processing CreateTablet for tablet 5b27df075eba4c6db0f45311133ff429 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:44.708282  1480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b27df075eba4c6db0f45311133ff429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.709712  1338 tablet_service.cc:1511] Processing CreateTablet for tablet fdeefaeba5c24b7a96d2413978ccc4e5 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:44.711560  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdeefaeba5c24b7a96d2413978ccc4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.712208  1479 tablet_service.cc:1511] Processing CreateTablet for tablet 78b7ca1ca11049b590e84b9ff588270a (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:44.714349  1479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b7ca1ca11049b590e84b9ff588270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.715932  1337 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1a51e5426f4076a4f6ecc9887779c3 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:44.718032  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1a51e5426f4076a4f6ecc9887779c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.737278  1336 tablet_service.cc:1511] Processing CreateTablet for tablet ade1be5869844d02a7e6cf1f195dc559 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:44.738873  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1be5869844d02a7e6cf1f195dc559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.743147  1335 tablet_service.cc:1511] Processing CreateTablet for tablet 577caaca97e64a5caa06f4d2d500db4c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:44.744984  1335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577caaca97e64a5caa06f4d2d500db4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.747100  1334 tablet_service.cc:1511] Processing CreateTablet for tablet bbd0f4c451664f72a1f75274abfb2496 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:44.748564  1334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd0f4c451664f72a1f75274abfb2496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.756546  1333 tablet_service.cc:1511] Processing CreateTablet for tablet edc1782517cd4d2cbe842d3ea68e364c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:44.758006  1333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc1782517cd4d2cbe842d3ea68e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.772038  1332 tablet_service.cc:1511] Processing CreateTablet for tablet 97ddf31261234c61853d015f42a162e9 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:44.774222  1332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ddf31261234c61853d015f42a162e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.789803  1331 tablet_service.cc:1511] Processing CreateTablet for tablet 5b27df075eba4c6db0f45311133ff429 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:44.791680  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b27df075eba4c6db0f45311133ff429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.792203  1537 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:44.803956  1330 tablet_service.cc:1511] Processing CreateTablet for tablet 78b7ca1ca11049b590e84b9ff588270a (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:44.805888  1330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b7ca1ca11049b590e84b9ff588270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.809978  1410 tablet_service.cc:1511] Processing CreateTablet for tablet 577caaca97e64a5caa06f4d2d500db4c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:44.811456  1413 tablet_service.cc:1511] Processing CreateTablet for tablet fdeefaeba5c24b7a96d2413978ccc4e5 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:44.812532  1409 tablet_service.cc:1511] Processing CreateTablet for tablet bbd0f4c451664f72a1f75274abfb2496 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:44.813690  1412 tablet_service.cc:1511] Processing CreateTablet for tablet 1c1a51e5426f4076a4f6ecc9887779c3 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:44.815932  1411 tablet_service.cc:1511] Processing CreateTablet for tablet ade1be5869844d02a7e6cf1f195dc559 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:44.818122  1408 tablet_service.cc:1511] Processing CreateTablet for tablet edc1782517cd4d2cbe842d3ea68e364c (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:44.819985  1537 tablet_bootstrap.cc:654] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.820318  1407 tablet_service.cc:1511] Processing CreateTablet for tablet 97ddf31261234c61853d015f42a162e9 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:44.822791  1406 tablet_service.cc:1511] Processing CreateTablet for tablet 5b27df075eba4c6db0f45311133ff429 (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:44.825513  1405 tablet_service.cc:1511] Processing CreateTablet for tablet 78b7ca1ca11049b590e84b9ff588270a (DEFAULT_TABLE table=test-workload [id=fdcb66c7306a49fea229d924645b6939]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:44.811484  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 577caaca97e64a5caa06f4d2d500db4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.828936  1408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc1782517cd4d2cbe842d3ea68e364c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.829806  1407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97ddf31261234c61853d015f42a162e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.833757  1405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b7ca1ca11049b590e84b9ff588270a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.836759  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdeefaeba5c24b7a96d2413978ccc4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.837381  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c1a51e5426f4076a4f6ecc9887779c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.841789  1406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b27df075eba4c6db0f45311133ff429. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.845825  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1be5869844d02a7e6cf1f195dc559. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.851241  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd0f4c451664f72a1f75274abfb2496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:44.870872  1540 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:44.881136  1538 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:44.890743  1540 tablet_bootstrap.cc:654] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.913353  1537 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:44.913935  1537 ts_tablet_manager.cc:1403] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.122s	user 0.036s	sys 0.038s
I20260709 12:17:44.917044  1537 raft_consensus.cc:359] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:44.917795  1537 raft_consensus.cc:385] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.918077  1537 raft_consensus.cc:740] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.918953  1537 consensus_queue.cc:260] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:44.934343  1538 tablet_bootstrap.cc:654] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.935209  1540 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:44.935856  1540 ts_tablet_manager.cc:1403] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.000s
I20260709 12:17:44.938896  1540 raft_consensus.cc:359] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:44.939822  1540 raft_consensus.cc:385] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.940171  1540 raft_consensus.cc:740] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.941080  1540 consensus_queue.cc:260] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:44.945354  1538 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:44.946019  1538 ts_tablet_manager.cc:1403] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.002s
I20260709 12:17:44.946965  1448 heartbeater.cc:499] Master 127.21.150.254:37815 was elected leader, sending a full tablet report...
I20260709 12:17:44.947400  1537 ts_tablet_manager.cc:1434] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.033s	user 0.013s	sys 0.019s
I20260709 12:17:44.947554  1540 ts_tablet_manager.cc:1434] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:17:44.948194  1522 heartbeater.cc:499] Master 127.21.150.254:37815 was elected leader, sending a full tablet report...
I20260709 12:17:44.949178  1540 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:44.949090  1538 raft_consensus.cc:359] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:44.949975  1538 raft_consensus.cc:385] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.950300  1538 raft_consensus.cc:740] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.951045  1538 consensus_queue.cc:260] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:44.952353  1537 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:44.959414  1537 tablet_bootstrap.cc:654] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.964942  1540 tablet_bootstrap.cc:654] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.969467  1373 heartbeater.cc:499] Master 127.21.150.254:37815 was elected leader, sending a full tablet report...
I20260709 12:17:44.969978  1538 ts_tablet_manager.cc:1434] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.016s
I20260709 12:17:44.970983  1537 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:44.970995  1538 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:44.971692  1537 ts_tablet_manager.cc:1403] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 12:17:44.974749  1537 raft_consensus.cc:359] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:44.975507  1537 raft_consensus.cc:385] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:44.975817  1537 raft_consensus.cc:740] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:44.976990  1537 consensus_queue.cc:260] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:44.978700  1538 tablet_bootstrap.cc:654] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.979332  1537 ts_tablet_manager.cc:1434] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 12:17:44.981019  1537 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:44.987875  1537 tablet_bootstrap.cc:654] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:44.990113  1540 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:44.991055  1540 ts_tablet_manager.cc:1403] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.000s
I20260709 12:17:44.996251  1542 raft_consensus.cc:493] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:44.997192  1542 raft_consensus.cc:515] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.007053  1538 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.007639  1538 ts_tablet_manager.cc:1403] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.023s
I20260709 12:17:45.007817  1537 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.008371  1537 ts_tablet_manager.cc:1403] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 12:17:45.009297  1542 leader_election.cc:290] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.010589  1538 raft_consensus.cc:359] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.013717  1538 raft_consensus.cc:385] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.014147  1538 raft_consensus.cc:740] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.015177  1538 consensus_queue.cc:260] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.012233  1537 raft_consensus.cc:359] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:44.995879  1540 raft_consensus.cc:359] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.017627  1537 raft_consensus.cc:385] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.017831  1540 raft_consensus.cc:385] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.017983  1537 raft_consensus.cc:740] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.018185  1540 raft_consensus.cc:740] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.018893  1537 consensus_queue.cc:260] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.021498  1537 ts_tablet_manager.cc:1434] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.001s
I20260709 12:17:45.022478  1537 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:45.025970  1538 ts_tablet_manager.cc:1434] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:17:45.027233  1538 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.026996  1540 consensus_queue.cc:260] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.029312  1540 ts_tablet_manager.cc:1434] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:17:45.033529  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.036116  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.036823  1348 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.036921  1540 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.037408  1537 tablet_bootstrap.cc:654] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:45.038499  1463 leader_election.cc:343] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.039286  1463 leader_election.cc:304] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:45.040141  1542 raft_consensus.cc:2804] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.040632  1542 raft_consensus.cc:493] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.041072  1542 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.044271  1540 tablet_bootstrap.cc:654] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.033828  1538 tablet_bootstrap.cc:654] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.062613  1540 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.063114  1540 ts_tablet_manager.cc:1403] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.004s
I20260709 12:17:45.063476  1538 tablet_bootstrap.cc:492] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.064044  1538 ts_tablet_manager.cc:1403] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.003s
I20260709 12:17:45.064435  1542 raft_consensus.cc:515] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.065549  1540 raft_consensus.cc:359] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.066315  1540 raft_consensus.cc:385] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.066613  1540 raft_consensus.cc:740] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.067027  1538 raft_consensus.cc:359] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.067463  1537 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.067884  1538 raft_consensus.cc:385] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.067404  1540 consensus_queue.cc:260] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.068423  1537 ts_tablet_manager.cc:1403] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.017s
I20260709 12:17:45.067492  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.068575  1538 raft_consensus.cc:740] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.069249  1348 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.070578  1540 ts_tablet_manager.cc:1434] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:45.072027  1537 raft_consensus.cc:359] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.072888  1537 raft_consensus.cc:385] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.073253  1537 raft_consensus.cc:740] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.073745  1540 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.074210  1537 consensus_queue.cc:260] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.079191  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
W20260709 12:17:45.081746  1463 leader_election.cc:343] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.082386  1540 tablet_bootstrap.cc:654] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.082501  1542 leader_election.cc:290] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.082420  1348 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:45.084632  1463 leader_election.cc:304] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:45.087477  1538 consensus_queue.cc:260] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.092729  1542 raft_consensus.cc:2804] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.093319  1542 raft_consensus.cc:697] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:45.094214  1542 consensus_queue.cc:237] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.096824  1538 ts_tablet_manager.cc:1434] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 12:17:45.100530  1537 ts_tablet_manager.cc:1434] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.032s	user 0.014s	sys 0.014s
I20260709 12:17:45.101851  1537 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:45.103324  1540 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.104307  1540 ts_tablet_manager.cc:1403] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.009s
I20260709 12:17:45.104573  1538 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.107762  1540 raft_consensus.cc:359] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.108395  1537 tablet_bootstrap.cc:654] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.108553  1540 raft_consensus.cc:385] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.108915  1540 raft_consensus.cc:740] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.109881  1540 consensus_queue.cc:260] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.113256  1540 ts_tablet_manager.cc:1434] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:17:45.114604  1540 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.120712  1537 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.121335  1537 ts_tablet_manager.cc:1403] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 12:17:45.121488  1540 tablet_bootstrap.cc:654] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.124717  1537 raft_consensus.cc:359] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.125716  1537 raft_consensus.cc:385] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.126083  1537 raft_consensus.cc:740] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.127373  1537 consensus_queue.cc:260] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.129263  1545 raft_consensus.cc:493] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.129868  1545 raft_consensus.cc:515] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.130677  1537 ts_tablet_manager.cc:1434] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:17:45.131758  1537 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:45.132403  1545 leader_election.cc:290] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.132777  1540 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.133324  1540 ts_tablet_manager.cc:1403] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:17:45.136863  1540 raft_consensus.cc:359] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.137300  1538 tablet_bootstrap.cc:654] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.137662  1540 raft_consensus.cc:385] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.138022  1540 raft_consensus.cc:740] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.138717  1537 tablet_bootstrap.cc:654] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.138743  1544 raft_consensus.cc:493] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.139446  1544 raft_consensus.cc:515] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.143321  1245 catalog_manager.cc:5697] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.145582  1540 consensus_queue.cc:260] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.146205  1544 leader_election.cc:290] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.165107  1540 ts_tablet_manager.cc:1434] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.001s
I20260709 12:17:45.166602  1540 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.168151  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.168932  1423 raft_consensus.cc:2468] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.171594  1542 raft_consensus.cc:493] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.172148  1542 raft_consensus.cc:515] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.172982  1538 tablet_bootstrap.cc:492] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.173731  1538 ts_tablet_manager.cc:1403] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.002s
I20260709 12:17:45.174366  1542 leader_election.cc:290] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.174607  1314 leader_election.cc:304] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.177619  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.178615  1348 raft_consensus.cc:2468] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.181123  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.181283  1463 leader_election.cc:304] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.181818  1423 raft_consensus.cc:2468] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.182282  1542 raft_consensus.cc:2804] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.182716  1542 raft_consensus.cc:493] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.183122  1542 raft_consensus.cc:3060] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.183425  1545 raft_consensus.cc:2804] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.184026  1545 raft_consensus.cc:493] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.184502  1545 raft_consensus.cc:3060] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:45.197788  1294 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:45.200547  1540 tablet_bootstrap.cc:654] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.200594  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.197386  1538 raft_consensus.cc:359] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.202808  1538 raft_consensus.cc:385] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.203132  1538 raft_consensus.cc:740] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.204069  1538 consensus_queue.cc:260] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.206230  1537 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.206812  1537 ts_tablet_manager.cc:1403] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.019s
I20260709 12:17:45.201534  1542 raft_consensus.cc:515] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.207643  1497 raft_consensus.cc:2393] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d972080851fd4e7784753234934d65bc in current term 1: Already voted for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in this term.
I20260709 12:17:45.208698  1545 raft_consensus.cc:515] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.211006  1537 raft_consensus.cc:359] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.213289  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.214979  1348 raft_consensus.cc:2393] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa3ca2b7e17e42c5becd0039a2ac0453 in current term 1: Already voted for candidate d972080851fd4e7784753234934d65bc in this term.
I20260709 12:17:45.213559  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.215596  1545 leader_election.cc:290] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.216298  1537 raft_consensus.cc:385] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:17:45.216812  1389 leader_election.cc:343] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.217242  1537 raft_consensus.cc:740] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.218056  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.218124  1537 consensus_queue.cc:260] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.218983  1423 raft_consensus.cc:3060] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.220474  1537 ts_tablet_manager.cc:1434] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:17:45.221213  1537 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:45.221807  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.222620  1497 raft_consensus.cc:2393] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d972080851fd4e7784753234934d65bc in current term 1: Already voted for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in this term.
I20260709 12:17:45.225853  1538 ts_tablet_manager.cc:1434] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.007s
I20260709 12:17:45.226975  1542 leader_election.cc:290] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.227198  1538 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.227973  1537 tablet_bootstrap.cc:654] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.229702  1423 raft_consensus.cc:2468] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.231928  1314 leader_election.cc:304] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:45.232081  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.232779  1497 raft_consensus.cc:2468] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 0.
I20260709 12:17:45.234486  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1be5869844d02a7e6cf1f195dc559" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.236079  1568 raft_consensus.cc:2804] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.237190  1568 raft_consensus.cc:697] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:45.235683  1423 raft_consensus.cc:2393] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa3ca2b7e17e42c5becd0039a2ac0453 in current term 1: Already voted for candidate d972080851fd4e7784753234934d65bc in this term.
I20260709 12:17:45.245662  1389 leader_election.cc:304] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc
I20260709 12:17:45.247665  1562 raft_consensus.cc:2804] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.248093  1562 raft_consensus.cc:493] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.248440  1562 raft_consensus.cc:3060] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.245638  1568 consensus_queue.cc:237] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.251500  1537 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.252081  1537 ts_tablet_manager.cc:1403] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.031s	user 0.001s	sys 0.020s
I20260709 12:17:45.253383  1463 leader_election.cc:304] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:45.254662  1542 raft_consensus.cc:2749] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.255766  1537 raft_consensus.cc:359] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.256860  1537 raft_consensus.cc:385] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.257228  1537 raft_consensus.cc:740] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.258719  1544 raft_consensus.cc:493] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.258468  1537 consensus_queue.cc:260] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.259286  1544 raft_consensus.cc:515] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.262552  1544 leader_election.cc:290] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.263326  1537 ts_tablet_manager.cc:1434] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.001s
I20260709 12:17:45.264509  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.264818  1537 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
W20260709 12:17:45.266170  1389 leader_election.cc:343] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.267472  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
W20260709 12:17:45.269173  1389 leader_election.cc:343] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.269761  1389 leader_election.cc:304] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463; no voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:45.270908  1544 raft_consensus.cc:2749] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.272073  1562 raft_consensus.cc:515] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.274510  1537 tablet_bootstrap.cc:654] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.275120  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.277698  1562 leader_election.cc:290] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
W20260709 12:17:45.278424  1389 leader_election.cc:343] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.281612  1538 tablet_bootstrap.cc:654] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.281602  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.282429  1497 raft_consensus.cc:3060] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.285507  1562 raft_consensus.cc:493] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.286281  1562 raft_consensus.cc:515] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.287577  1245 catalog_manager.cc:5697] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.290470  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.290750  1562 leader_election.cc:290] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.291253  1347 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 0.
I20260709 12:17:45.294348  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.295099  1496 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 0.
I20260709 12:17:45.295982  1540 tablet_bootstrap.cc:492] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.296736  1540 ts_tablet_manager.cc:1403] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.018s
I20260709 12:17:45.301052  1540 raft_consensus.cc:359] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.302002  1540 raft_consensus.cc:385] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.302361  1540 raft_consensus.cc:740] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.303310  1540 consensus_queue.cc:260] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.293001  1389 leader_election.cc:304] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.304522  1544 raft_consensus.cc:2804] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.304899  1544 raft_consensus.cc:493] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.305235  1544 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.314203  1540 ts_tablet_manager.cc:1434] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 12:17:45.315734  1544 raft_consensus.cc:515] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.321345  1497 raft_consensus.cc:2468] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:45.322655  1389 leader_election.cc:304] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc
I20260709 12:17:45.323479  1562 raft_consensus.cc:2804] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.329370  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.330240  1540 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.330196  1347 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.334010  1537 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.334370  1538 tablet_bootstrap.cc:492] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.334697  1537 ts_tablet_manager.cc:1403] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.050s
I20260709 12:17:45.335006  1538 ts_tablet_manager.cc:1403] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.108s	user 0.032s	sys 0.022s
I20260709 12:17:45.330868  1544 leader_election.cc:290] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.337862  1537 raft_consensus.cc:359] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.338660  1537 raft_consensus.cc:385] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.338304  1538 raft_consensus.cc:359] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.339063  1540 tablet_bootstrap.cc:654] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.339293  1538 raft_consensus.cc:385] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.339762  1538 raft_consensus.cc:740] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.340610  1538 consensus_queue.cc:260] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.341032  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.341909  1497 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.343827  1538 ts_tablet_manager.cc:1434] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:45.345376  1538 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.338999  1537 raft_consensus.cc:740] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.350821  1537 consensus_queue.cc:260] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.351454  1497 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:45.353096  1389 leader_election.cc:304] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.354059  1544 raft_consensus.cc:2804] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.354421  1537 ts_tablet_manager.cc:1434] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:17:45.354521  1544 raft_consensus.cc:697] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:45.356024  1537 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:45.356015  1544 consensus_queue.cc:237] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.364846  1542 raft_consensus.cc:493] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.366047  1542 raft_consensus.cc:515] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.369689  1542 leader_election.cc:290] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.370440  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
W20260709 12:17:45.372606  1463 leader_election.cc:343] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.374104  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
W20260709 12:17:45.375871  1463 leader_election.cc:343] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:45.376345  1463 leader_election.cc:304] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:45.377446  1537 tablet_bootstrap.cc:654] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.377497  1542 raft_consensus.cc:2749] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.384949  1538 tablet_bootstrap.cc:654] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.386401  1562 raft_consensus.cc:697] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:45.386736  1347 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:45.387540  1562 consensus_queue.cc:237] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.391851  1537 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:45.392702  1537 ts_tablet_manager.cc:1403] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.000s
I20260709 12:17:45.396253  1537 raft_consensus.cc:359] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.397087  1537 raft_consensus.cc:385] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.397392  1537 raft_consensus.cc:740] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.398707  1540 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.398433  1537 consensus_queue.cc:260] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.399302  1538 tablet_bootstrap.cc:492] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.399631  1540 ts_tablet_manager.cc:1403] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.077s	user 0.030s	sys 0.036s
I20260709 12:17:45.399923  1538 ts_tablet_manager.cc:1403] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.055s	user 0.017s	sys 0.000s
I20260709 12:17:45.401733  1537 ts_tablet_manager.cc:1434] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:17:45.403352  1540 raft_consensus.cc:359] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.404167  1540 raft_consensus.cc:385] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.404436  1540 raft_consensus.cc:740] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.405143  1540 consensus_queue.cc:260] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.407181  1540 ts_tablet_manager.cc:1434] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:45.407996  1540 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.407593  1243 catalog_manager.cc:5697] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.412523  1538 raft_consensus.cc:359] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.413281  1538 raft_consensus.cc:385] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.413582  1538 raft_consensus.cc:740] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.414264  1538 consensus_queue.cc:260] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.414633  1540 tablet_bootstrap.cc:654] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.416812  1538 ts_tablet_manager.cc:1434] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:17:45.419265  1538 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.426200  1538 tablet_bootstrap.cc:654] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.426783  1245 catalog_manager.cc:5697] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.431442  1540 tablet_bootstrap.cc:492] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.431974  1540 ts_tablet_manager.cc:1403] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.015s
I20260709 12:17:45.432480  1538 tablet_bootstrap.cc:492] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.433043  1538 ts_tablet_manager.cc:1403] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:17:45.434406  1540 raft_consensus.cc:359] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.435088  1540 raft_consensus.cc:385] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.435321  1540 raft_consensus.cc:740] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.435900  1538 raft_consensus.cc:359] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.436126  1540 consensus_queue.cc:260] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.436663  1538 raft_consensus.cc:385] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.437058  1538 raft_consensus.cc:740] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.438370  1540 ts_tablet_manager.cc:1434] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:45.437994  1538 consensus_queue.cc:260] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.439530  1540 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:45.440815  1538 ts_tablet_manager.cc:1434] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:17:45.442057  1538 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.445015  1540 tablet_bootstrap.cc:654] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.449995  1538 tablet_bootstrap.cc:654] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.452894  1540 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:45.453372  1540 ts_tablet_manager.cc:1403] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:17:45.454584  1538 tablet_bootstrap.cc:492] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.455035  1538 ts_tablet_manager.cc:1403] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 12:17:45.455541  1540 raft_consensus.cc:359] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.456096  1540 raft_consensus.cc:385] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.456326  1540 raft_consensus.cc:740] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.456924  1540 consensus_queue.cc:260] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.457175  1538 raft_consensus.cc:359] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.457790  1538 raft_consensus.cc:385] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.458060  1538 raft_consensus.cc:740] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.458863  1540 ts_tablet_manager.cc:1434] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:17:45.458737  1538 consensus_queue.cc:260] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.461167  1538 ts_tablet_manager.cc:1434] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:17:45.462162  1538 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:45.467347  1568 raft_consensus.cc:493] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.468479  1538 tablet_bootstrap.cc:654] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:45.468327  1568 raft_consensus.cc:515] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.470731  1568 leader_election.cc:290] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.471944  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.472646  1497 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.472699  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.473477  1423 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.473971  1314 leader_election.cc:304] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.475718  1545 raft_consensus.cc:2804] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.476027  1568 raft_consensus.cc:493] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.476081  1545 raft_consensus.cc:493] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.476643  1545 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.476572  1568 raft_consensus.cc:515] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.477025  1538 tablet_bootstrap.cc:492] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:45.478104  1538 ts_tablet_manager.cc:1403] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:17:45.479651  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.480911  1538 raft_consensus.cc:359] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.481671  1538 raft_consensus.cc:385] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:45.481877  1568 leader_election.cc:290] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.481959  1538 raft_consensus.cc:740] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:45.483075  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc1782517cd4d2cbe842d3ea68e364c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.482885  1538 consensus_queue.cc:260] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.483807  1497 raft_consensus.cc:2393] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d972080851fd4e7784753234934d65bc in current term 1: Already voted for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in this term.
I20260709 12:17:45.484635  1545 raft_consensus.cc:515] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.485080  1538 ts_tablet_manager.cc:1434] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:45.485148  1314 leader_election.cc:304] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d972080851fd4e7784753234934d65bc; no voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:45.486801  1568 raft_consensus.cc:3060] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.486891  1545 leader_election.cc:290] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.494904  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.495635  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.495668  1423 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.496330  1497 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.497121  1568 raft_consensus.cc:2749] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.501992  1423 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.502338  1497 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.503137  1314 leader_election.cc:304] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.503845  1568 raft_consensus.cc:2804] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.504257  1568 raft_consensus.cc:697] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:45.504911  1568 consensus_queue.cc:237] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.513198  1568 raft_consensus.cc:493] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.514037  1568 raft_consensus.cc:515] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.514357  1244 catalog_manager.cc:5697] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.516364  1568 leader_election.cc:290] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.517264  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.517457  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.518056  1423 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.518117  1497 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.519272  1314 leader_election.cc:304] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.519928  1568 raft_consensus.cc:2804] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.520289  1568 raft_consensus.cc:493] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.520617  1568 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.526661  1568 raft_consensus.cc:515] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.528540  1568 leader_election.cc:290] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.529574  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.530285  1423 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.529917  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.530951  1497 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.536173  1423 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.536346  1497 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.537454  1314 leader_election.cc:304] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.538175  1568 raft_consensus.cc:2804] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.538604  1568 raft_consensus.cc:697] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:45.539382  1568 consensus_queue.cc:237] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.546099  1244 catalog_manager.cc:5697] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.555307  1542 raft_consensus.cc:493] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.555845  1542 raft_consensus.cc:515] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.557487  1542 leader_election.cc:290] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.558243  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.558467  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.558835  1347 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.559135  1423 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.559899  1463 leader_election.cc:304] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.560606  1542 raft_consensus.cc:2804] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.561003  1542 raft_consensus.cc:493] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.561388  1542 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.566881  1542 raft_consensus.cc:515] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.568514  1542 leader_election.cc:290] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.569423  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.569675  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.570134  1347 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.570251  1423 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.577507  1347 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:45.577512  1423 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:45.578853  1463 leader_election.cc:304] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.579559  1542 raft_consensus.cc:2804] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.580040  1542 raft_consensus.cc:697] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:45.580868  1542 consensus_queue.cc:237] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.582168  1562 raft_consensus.cc:493] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.582635  1562 raft_consensus.cc:515] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:45.584681  1562 leader_election.cc:290] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251), d972080851fd4e7784753234934d65bc (127.21.150.193:45907)
I20260709 12:17:45.585963  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.586009  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.586952  1347 raft_consensus.cc:2393] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3d8704396ba4bbcb6a0a52cf4a79463 in current term 1: Already voted for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in this term.
I20260709 12:17:45.587875  1389 leader_election.cc:304] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463; no voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:45.588729  1562 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.590615  1244 catalog_manager.cc:5697] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.595351  1562 raft_consensus.cc:2749] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:45.599445  1542 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:45.600340  1554 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:45.678332  1568 consensus_queue.cc:1048] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.681283  1554 raft_consensus.cc:493] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.681806  1554 raft_consensus.cc:515] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.683928  1554 leader_election.cc:290] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.685163  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b7ca1ca11049b590e84b9ff588270a" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:45.685549  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b7ca1ca11049b590e84b9ff588270a" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.686740  1347 raft_consensus.cc:2468] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.686251  1423 raft_consensus.cc:2468] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:45.688148  1463 leader_election.cc:304] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.690070  1554 raft_consensus.cc:2804] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.690480  1554 raft_consensus.cc:493] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.690599  1586 consensus_queue.cc:1048] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:45.690896  1554 raft_consensus.cc:3060] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.690927  1545 raft_consensus.cc:493] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:45.697773  1545 raft_consensus.cc:515] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.699915  1554 raft_consensus.cc:515] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.701150  1545 leader_election.cc:290] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.702229  1554 leader_election.cc:290] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:45.702412  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:45.702631  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:45.703218  1423 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.704473  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b7ca1ca11049b590e84b9ff588270a" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:45.705121  1347 raft_consensus.cc:3060] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.705739  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78b7ca1ca11049b590e84b9ff588270a" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.706360  1423 raft_consensus.cc:3060] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.703279  1497 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:45.704703  1314 leader_election.cc:304] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.710727  1586 raft_consensus.cc:2804] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:45.711103  1586 raft_consensus.cc:493] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:45.711467  1586 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.713011  1347 raft_consensus.cc:2468] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:45.714361  1463 leader_election.cc:304] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:45.715406  1554 raft_consensus.cc:2804] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.716663  1554 raft_consensus.cc:697] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:45.717157  1423 raft_consensus.cc:2468] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:45.717545  1554 consensus_queue.cc:237] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.719326  1586 raft_consensus.cc:515] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.725642  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:45.725725  1586 leader_election.cc:290] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:45.725739  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:45.726243  1497 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.726537  1423 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:45.730737  1244 catalog_manager.cc:5697] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:45.732618  1423 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.733502  1497 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:45.733778  1314 leader_election.cc:304] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:45.734880  1586 raft_consensus.cc:2804] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:45.735357  1586 raft_consensus.cc:697] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:45.736128  1586 consensus_queue.cc:237] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:45.743980  1244 catalog_manager.cc:5697] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:45.811285  1562 consensus_queue.cc:1048] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.819326  1575 consensus_queue.cc:1048] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:45.822043  1562 consensus_queue.cc:1048] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.836191  1562 consensus_queue.cc:1048] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.925876  1586 consensus_queue.cc:1048] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.936718  1586 consensus_queue.cc:1048] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:45.957240  1497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
mode: GRACEFUL
new_leader_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:46642
I20260709 12:17:45.957827  1497 raft_consensus.cc:606] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Received request to transfer leadership to TS d972080851fd4e7784753234934d65bc
I20260709 12:17:45.970597  1423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
mode: GRACEFUL
new_leader_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:35548
I20260709 12:17:45.971117  1423 raft_consensus.cc:606] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Received request to transfer leadership to TS d972080851fd4e7784753234934d65bc
I20260709 12:17:45.979118  1497 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
mode: GRACEFUL
new_leader_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:46642
I20260709 12:17:45.979640  1497 raft_consensus.cc:606] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Received request to transfer leadership to TS d972080851fd4e7784753234934d65bc
I20260709 12:17:45.993395  1347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:52284
I20260709 12:17:45.993954  1347 raft_consensus.cc:606] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:46.001885  1347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97ddf31261234c61853d015f42a162e9"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:52284
I20260709 12:17:46.002410  1347 raft_consensus.cc:606] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:46.007102  1347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:52284
I20260709 12:17:46.007736  1347 raft_consensus.cc:606] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:46.010160  1586 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.021581  1568 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.033823  1612 raft_consensus.cc:993] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: : Instructing follower f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.034380  1545 raft_consensus.cc:1081] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.035899  1423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:35536
I20260709 12:17:46.036397  1423 raft_consensus.cc:493] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.036695  1423 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.041949  1423 raft_consensus.cc:515] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.043558  1423 leader_election.cc:290] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:46.044541  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:46.044826  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.045125  1347 raft_consensus.cc:3055] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.045513  1347 raft_consensus.cc:740] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.045571  1497 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.046317  1347 consensus_queue.cc:260] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.047640  1347 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.052081  1497 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.053192  1389 leader_election.cc:304] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:46.053516  1347 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.054004  1562 raft_consensus.cc:2804] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.054499  1562 raft_consensus.cc:697] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:46.055245  1562 consensus_queue.cc:237] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.061470  1245 catalog_manager.cc:5697] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.116046  1554 consensus_queue.cc:1048] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.128232  1554 consensus_queue.cc:1048] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.129626  1592 raft_consensus.cc:993] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: : Instructing follower d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.130192  1592 raft_consensus.cc:1081] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Signalling peer d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.132001  1348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5"
dest_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:52272
I20260709 12:17:46.132616  1348 raft_consensus.cc:493] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.133049  1348 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.137812  1554 raft_consensus.cc:993] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: : Instructing follower d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.138363  1615 raft_consensus.cc:1081] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Signalling peer d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.140064  1347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c"
dest_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:52272
I20260709 12:17:46.140641  1347 raft_consensus.cc:493] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.141016  1347 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.140825  1348 raft_consensus.cc:515] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:46.143481  1348 leader_election.cc:290] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Requested vote from peers fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:46.144980  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.145641  1497 raft_consensus.cc:3055] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.145762  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:46.146026  1497 raft_consensus.cc:740] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:46.146473  1423 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.146914  1497 consensus_queue.cc:260] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:46.148281  1497 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.148334  1347 raft_consensus.cc:515] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.151202  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:46.151763  1347 leader_election.cc:290] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:46.152113  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "577caaca97e64a5caa06f4d2d500db4c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.152683  1496 raft_consensus.cc:3055] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.153009  1496 raft_consensus.cc:740] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:46.153136  1423 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.153870  1496 consensus_queue.cc:260] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.154465  1314 leader_election.cc:304] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:46.155115  1496 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.155391  1545 raft_consensus.cc:2804] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.155889  1545 raft_consensus.cc:697] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.156715  1545 consensus_queue.cc:237] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:46.151774  1422 raft_consensus.cc:3060] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.161674  1496 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.162881  1314 leader_election.cc:304] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:46.163740  1612 raft_consensus.cc:2804] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.164224  1612 raft_consensus.cc:697] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.165064  1612 consensus_queue.cc:237] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.164989  1245 catalog_manager.cc:5697] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 1 to 2, leader changed from fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195) to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 2 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.167373  1497 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.168004  1422 raft_consensus.cc:2468] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.178175  1245 catalog_manager.cc:5697] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 1 to 2, leader changed from fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195) to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 2 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.226071  1545 consensus_queue.cc:1048] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.232722  1544 raft_consensus.cc:993] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: : Instructing follower d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.233207  1544 raft_consensus.cc:1081] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Signalling peer d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:46.236901  1347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3"
dest_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:52278
I20260709 12:17:46.237515  1347 raft_consensus.cc:493] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.237943  1347 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.238637  1586 consensus_queue.cc:1048] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.248085  1347 raft_consensus.cc:515] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.251060  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:46.251225  1347 leader_election.cc:290] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:46.251675  1423 raft_consensus.cc:3055] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.252050  1423 raft_consensus.cc:740] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:46.252724  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c1a51e5426f4076a4f6ecc9887779c3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.252866  1423 consensus_queue.cc:260] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.253728  1496 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.254419  1423 raft_consensus.cc:3060] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.259567  1618 raft_consensus.cc:993] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: : Instructing follower f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.260090  1618 raft_consensus.cc:1081] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.261984  1422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:35536
I20260709 12:17:46.262419  1496 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.262586  1422 raft_consensus.cc:493] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.262629  1423 raft_consensus.cc:2468] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:46.263096  1422 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.263816  1314 leader_election.cc:304] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:46.264582  1586 raft_consensus.cc:2804] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.265112  1586 raft_consensus.cc:697] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.265993  1586 consensus_queue.cc:237] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.270731  1422 raft_consensus.cc:515] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.272725  1422 leader_election.cc:290] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:46.274224  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.274323  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b27df075eba4c6db0f45311133ff429" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:46.273633  1245 catalog_manager.cc:5697] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 1 to 2, leader changed from f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194) to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 2 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.274896  1496 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.275113  1347 raft_consensus.cc:3055] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.275590  1347 raft_consensus.cc:740] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.276553  1347 consensus_queue.cc:260] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.277896  1347 raft_consensus.cc:3060] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.282295  1496 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.283509  1389 leader_election.cc:304] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:46.284301  1544 raft_consensus.cc:2804] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.284749  1544 raft_consensus.cc:697] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:46.285589  1347 raft_consensus.cc:2468] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.285516  1544 consensus_queue.cc:237] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.292104  1245 catalog_manager.cc:5697] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.306803  1615 consensus_queue.cc:1048] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.318187  1554 consensus_queue.cc:1048] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:46.462359  1545 raft_consensus.cc:993] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: : Instructing follower f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.462826  1545 raft_consensus.cc:1081] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:46.464363  1422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97ddf31261234c61853d015f42a162e9"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:35536
I20260709 12:17:46.464880  1422 raft_consensus.cc:493] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:46.465211  1422 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.470640  1422 raft_consensus.cc:515] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.472282  1422 leader_election.cc:290] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:46.473202  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:46.473533  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97ddf31261234c61853d015f42a162e9" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:46.473943  1347 raft_consensus.cc:3055] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:46.474078  1496 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.474329  1347 raft_consensus.cc:740] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:46.475306  1347 consensus_queue.cc:260] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.476557  1347 raft_consensus.cc:3060] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:46.481696  1496 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.483003  1389 leader_election.cc:304] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:46.483626  1347 raft_consensus.cc:2468] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:46.483750  1622 raft_consensus.cc:2804] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:46.484488  1622 raft_consensus.cc:697] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:46.485297  1622 consensus_queue.cc:237] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:46.491977  1244 catalog_manager.cc:5697] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:46.573561  1496 raft_consensus.cc:1275] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.575089  1545 consensus_queue.cc:1048] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.583747  1497 raft_consensus.cc:1275] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.585736  1618 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.587736  1422 raft_consensus.cc:1275] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.589253  1618 consensus_queue.cc:1048] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.596235  1422 raft_consensus.cc:1275] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.600195  1612 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.608369  1347 raft_consensus.cc:1275] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.609939  1622 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.618309  1497 raft_consensus.cc:1275] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.620812  1622 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:46.665198  1347 raft_consensus.cc:1275] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.666566  1622 consensus_queue.cc:1048] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.674608  1497 raft_consensus.cc:1275] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.676360  1622 consensus_queue.cc:1048] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:46.791740  1497 raft_consensus.cc:1275] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.793156  1612 consensus_queue.cc:1048] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.800912  1422 raft_consensus.cc:1275] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.802361  1545 consensus_queue.cc:1048] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.966131  1497 raft_consensus.cc:1275] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.967475  1623 consensus_queue.cc:1048] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:46.968881  1347 raft_consensus.cc:1275] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:46.970219  1623 consensus_queue.cc:1048] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:49.043022  1245 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60764:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:49.046039  1245 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:49.102183  1332 tablet_service.cc:1511] Processing CreateTablet for tablet a5f2c8915f004e6499561957157d6fd3 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:49.103296  1407 tablet_service.cc:1511] Processing CreateTablet for tablet a5f2c8915f004e6499561957157d6fd3 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:49.104274  1332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f2c8915f004e6499561957157d6fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.105093  1407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f2c8915f004e6499561957157d6fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.104744  1483 tablet_service.cc:1511] Processing CreateTablet for tablet a5f2c8915f004e6499561957157d6fd3 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:49.106388  1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5f2c8915f004e6499561957157d6fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.111867  1330 tablet_service.cc:1511] Processing CreateTablet for tablet cea47c15c0e64fd2868d4a3bf2466eed (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:49.113639  1330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea47c15c0e64fd2868d4a3bf2466eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.117406  1480 tablet_service.cc:1511] Processing CreateTablet for tablet 7494b6f417b94ffb8d1fb53b87c4f28c (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:49.118681  1479 tablet_service.cc:1511] Processing CreateTablet for tablet f62a468441f34f06bfca60fa9f63ac6b (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:49.121346  1413 tablet_service.cc:1511] Processing CreateTablet for tablet 7494b6f417b94ffb8d1fb53b87c4f28c (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:49.127954  1413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7494b6f417b94ffb8d1fb53b87c4f28c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.121680  1409 tablet_service.cc:1511] Processing CreateTablet for tablet cea47c15c0e64fd2868d4a3bf2466eed (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:49.124363  1486 tablet_service.cc:1511] Processing CreateTablet for tablet bbd1a62cfa2748a1ba0ba2a91dce8286 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:49.131547  1409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea47c15c0e64fd2868d4a3bf2466eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.131474  1482 tablet_service.cc:1511] Processing CreateTablet for tablet a1315022c36141fbb8a5defbda0e4168 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:49.123040  1405 tablet_service.cc:1511] Processing CreateTablet for tablet f62a468441f34f06bfca60fa9f63ac6b (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:49.133293  1406 tablet_service.cc:1511] Processing CreateTablet for tablet bbd1a62cfa2748a1ba0ba2a91dce8286 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:49.134371  1405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62a468441f34f06bfca60fa9f63ac6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.135912  1410 tablet_service.cc:1511] Processing CreateTablet for tablet a1315022c36141fbb8a5defbda0e4168 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:49.136088  1335 tablet_service.cc:1511] Processing CreateTablet for tablet f62a468441f34f06bfca60fa9f63ac6b (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:49.136281  1333 tablet_service.cc:1511] Processing CreateTablet for tablet 7494b6f417b94ffb8d1fb53b87c4f28c (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:49.138340  1338 tablet_service.cc:1511] Processing CreateTablet for tablet a1315022c36141fbb8a5defbda0e4168 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:49.113633  1484 tablet_service.cc:1511] Processing CreateTablet for tablet cea47c15c0e64fd2868d4a3bf2466eed (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:49.138839  1331 tablet_service.cc:1511] Processing CreateTablet for tablet bbd1a62cfa2748a1ba0ba2a91dce8286 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:49.140393  1406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd1a62cfa2748a1ba0ba2a91dce8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.141175  1410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1315022c36141fbb8a5defbda0e4168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.133210  1480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7494b6f417b94ffb8d1fb53b87c4f28c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.142472  1484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cea47c15c0e64fd2868d4a3bf2466eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.145027  1412 tablet_service.cc:1511] Processing CreateTablet for tablet c544e2d317664736af713794ab7fb985 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:49.147435  1337 tablet_service.cc:1511] Processing CreateTablet for tablet c544e2d317664736af713794ab7fb985 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:49.148145  1412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c544e2d317664736af713794ab7fb985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.149897  1411 tablet_service.cc:1511] Processing CreateTablet for tablet 1826dbb6bf8142c9a1dce2e574f75d6d (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:49.152228  1482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1315022c36141fbb8a5defbda0e4168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.152163  1336 tablet_service.cc:1511] Processing CreateTablet for tablet 1826dbb6bf8142c9a1dce2e574f75d6d (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:49.153841  1335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62a468441f34f06bfca60fa9f63ac6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.154615  1336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1826dbb6bf8142c9a1dce2e574f75d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.155311  1333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7494b6f417b94ffb8d1fb53b87c4f28c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.156616  1408 tablet_service.cc:1511] Processing CreateTablet for tablet df7058eb8b8e4c4e916d5c8368307ebe (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:49.156898  1487 tablet_service.cc:1511] Processing CreateTablet for tablet c544e2d317664736af713794ab7fb985 (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:49.157793  1338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1315022c36141fbb8a5defbda0e4168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.158329  1487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c544e2d317664736af713794ab7fb985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.162107  1334 tablet_service.cc:1511] Processing CreateTablet for tablet df7058eb8b8e4c4e916d5c8368307ebe (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:49.162084  1479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f62a468441f34f06bfca60fa9f63ac6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.162084  1331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd1a62cfa2748a1ba0ba2a91dce8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.167559  1486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd1a62cfa2748a1ba0ba2a91dce8286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.167964  1334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df7058eb8b8e4c4e916d5c8368307ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.170217  1485 tablet_service.cc:1511] Processing CreateTablet for tablet 1826dbb6bf8142c9a1dce2e574f75d6d (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:49.170523  1481 tablet_service.cc:1511] Processing CreateTablet for tablet df7058eb8b8e4c4e916d5c8368307ebe (DEFAULT_TABLE table=soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:49.173993  1485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1826dbb6bf8142c9a1dce2e574f75d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.178028  1481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df7058eb8b8e4c4e916d5c8368307ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.185837  1411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1826dbb6bf8142c9a1dce2e574f75d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.187811  1337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c544e2d317664736af713794ab7fb985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.203418  1408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df7058eb8b8e4c4e916d5c8368307ebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:49.214136  1640 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.219084  1641 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.221859  1640 tablet_bootstrap.cc:654] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:49.241412  1294 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:49.284582  1641 tablet_bootstrap.cc:654] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.356472  1643 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.359022  1640 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.359619  1640 ts_tablet_manager.cc:1403] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.146s	user 0.009s	sys 0.018s
I20260709 12:17:49.362483  1640 raft_consensus.cc:359] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.363214  1640 raft_consensus.cc:385] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.363538  1640 raft_consensus.cc:740] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.364383  1640 consensus_queue.cc:260] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.365904  1641 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.366580  1641 ts_tablet_manager.cc:1403] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.148s	user 0.010s	sys 0.013s
I20260709 12:17:49.367650  1640 ts_tablet_manager.cc:1434] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:17:49.368757  1640 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.369480  1643 tablet_bootstrap.cc:654] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.370249  1641 raft_consensus.cc:359] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.371073  1641 raft_consensus.cc:385] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.371441  1641 raft_consensus.cc:740] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.372479  1641 consensus_queue.cc:260] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.387974  1640 tablet_bootstrap.cc:654] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.394668  1641 ts_tablet_manager.cc:1434] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.028s	user 0.012s	sys 0.014s
I20260709 12:17:49.396135  1641 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.395908  1643 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.397991  1643 ts_tablet_manager.cc:1403] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.000s
I20260709 12:17:49.402148  1643 raft_consensus.cc:359] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.402969  1643 raft_consensus.cc:385] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.403270  1643 raft_consensus.cc:740] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.404847  1641 tablet_bootstrap.cc:654] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.406384  1640 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.406944  1640 ts_tablet_manager.cc:1403] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.002s
I20260709 12:17:49.409754  1642 raft_consensus.cc:493] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.410414  1642 raft_consensus.cc:515] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.414613  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.415947  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
W20260709 12:17:49.417291  1314 leader_election.cc:343] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.417279  1641 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.418503  1641 ts_tablet_manager.cc:1403] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:17:49.420996  1642 leader_election.cc:290] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
W20260709 12:17:49.421460  1314 leader_election.cc:343] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.421921  1314 leader_election.cc:304] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d972080851fd4e7784753234934d65bc; no voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.423296  1642 raft_consensus.cc:2749] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:49.424057  1640 raft_consensus.cc:359] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.424866  1640 raft_consensus.cc:385] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.425184  1640 raft_consensus.cc:740] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.424314  1643 consensus_queue.cc:260] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.425951  1640 consensus_queue.cc:260] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.426431  1641 raft_consensus.cc:359] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.427323  1641 raft_consensus.cc:385] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.427707  1641 raft_consensus.cc:740] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.428797  1643 ts_tablet_manager.cc:1434] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:17:49.428586  1641 consensus_queue.cc:260] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.429961  1643 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.431257  1641 ts_tablet_manager.cc:1434] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:17:49.433174  1640 ts_tablet_manager.cc:1434] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:17:49.435211  1640 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: Bootstrap starting.
W20260709 12:17:49.440647  1389 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.193:45907, user_credentials={real_user=slave}} blocked reactor thread for 65192.1us
I20260709 12:17:49.434211  1641 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.443310  1625 raft_consensus.cc:493] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.443773  1625 raft_consensus.cc:515] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.446054  1625 leader_election.cc:290] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.448448  1641 tablet_bootstrap.cc:654] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.448773  1643 tablet_bootstrap.cc:654] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.448968  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.450428  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.451121  1423 raft_consensus.cc:2468] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:49.451380  1640 tablet_bootstrap.cc:654] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:49.452127  1463 leader_election.cc:343] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.453773  1463 leader_election.cc:304] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc
I20260709 12:17:49.455003  1643 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.454963  1625 raft_consensus.cc:2804] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.455478  1625 raft_consensus.cc:493] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.455652  1643 ts_tablet_manager.cc:1403] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.002s
I20260709 12:17:49.455850  1625 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.459043  1643 raft_consensus.cc:359] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.459762  1643 raft_consensus.cc:385] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.460039  1643 raft_consensus.cc:740] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.460949  1643 consensus_queue.cc:260] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.463735  1641 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.464288  1641 ts_tablet_manager.cc:1403] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.008s
I20260709 12:17:49.464444  1640 tablet_bootstrap.cc:492] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.465056  1640 ts_tablet_manager.cc:1403] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.005s
I20260709 12:17:49.467494  1641 raft_consensus.cc:359] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.468271  1641 raft_consensus.cc:385] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.468729  1641 raft_consensus.cc:740] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.468050  1640 raft_consensus.cc:359] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.469260  1640 raft_consensus.cc:385] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.469780  1640 raft_consensus.cc:740] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.469559  1641 consensus_queue.cc:260] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.470685  1640 consensus_queue.cc:260] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.472051  1641 ts_tablet_manager.cc:1434] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:49.473120  1641 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.474715  1640 ts_tablet_manager.cc:1434] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:17:49.475891  1640 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.476190  1643 ts_tablet_manager.cc:1434] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.012s
I20260709 12:17:49.477167  1643 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.482908  1640 tablet_bootstrap.cc:654] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.483937  1643 tablet_bootstrap.cc:654] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.470036  1625 raft_consensus.cc:515] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.491956  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:49.492636  1348 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.492892  1623 raft_consensus.cc:493] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.493615  1623 raft_consensus.cc:515] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.494549  1641 tablet_bootstrap.cc:654] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.497023  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.497584  1640 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.498111  1640 ts_tablet_manager.cc:1403] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.000s
W20260709 12:17:49.498910  1389 leader_election.cc:343] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.500061  1623 leader_election.cc:290] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.501581  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.501346  1640 raft_consensus.cc:359] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.502235  1640 raft_consensus.cc:385] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.502527  1640 raft_consensus.cc:740] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
W20260709 12:17:49.503262  1389 leader_election.cc:343] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.503960  1348 raft_consensus.cc:2468] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:49.503888  1389 leader_election.cc:304] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463; no voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.505637  1623 raft_consensus.cc:2749] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:49.507279  1642 raft_consensus.cc:493] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.507674  1463 leader_election.cc:304] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:49.507977  1642 raft_consensus.cc:515] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.509016  1654 raft_consensus.cc:2804] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.511831  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.513212  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
W20260709 12:17:49.514001  1314 leader_election.cc:343] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:49.515064  1314 leader_election.cc:343] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.515638  1314 leader_election.cc:304] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d972080851fd4e7784753234934d65bc; no voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.516929  1618 raft_consensus.cc:2749] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:49.518347  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:49.519151  1625 leader_election.cc:290] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.519237  1423 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.503288  1640 consensus_queue.cc:260] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.515087  1642 leader_election.cc:290] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.526918  1640 ts_tablet_manager.cc:1434] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:17:49.529994  1640 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.532668  1423 raft_consensus.cc:2468] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:49.537501  1654 raft_consensus.cc:697] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:49.538494  1654 consensus_queue.cc:237] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.543543  1643 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.543948  1641 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.544111  1643 ts_tablet_manager.cc:1403] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.067s	user 0.028s	sys 0.015s
I20260709 12:17:49.544497  1641 ts_tablet_manager.cc:1403] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.072s	user 0.022s	sys 0.013s
I20260709 12:17:49.547446  1643 raft_consensus.cc:359] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.547622  1641 raft_consensus.cc:359] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.548207  1643 raft_consensus.cc:385] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.548424  1641 raft_consensus.cc:385] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.548619  1643 raft_consensus.cc:740] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.548816  1641 raft_consensus.cc:740] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.549494  1640 tablet_bootstrap.cc:654] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.549649  1641 consensus_queue.cc:260] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.549623  1643 consensus_queue.cc:260] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.552273  1643 ts_tablet_manager.cc:1434] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:17:49.552578  1641 ts_tablet_manager.cc:1434] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:17:49.553323  1643 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.553802  1641 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.561076  1641 tablet_bootstrap.cc:654] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.563700  1643 tablet_bootstrap.cc:654] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.573734  1641 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.574342  1641 ts_tablet_manager.cc:1403] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 12:17:49.574415  1643 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.574947  1643 ts_tablet_manager.cc:1403] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 12:17:49.573760  1640 tablet_bootstrap.cc:492] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.577029  1640 ts_tablet_manager.cc:1403] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.014s
I20260709 12:17:49.577395  1643 raft_consensus.cc:359] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.576973  1245 catalog_manager.cc:5697] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:49.578218  1643 raft_consensus.cc:385] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.577790  1641 raft_consensus.cc:359] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.578611  1643 raft_consensus.cc:740] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.578755  1641 raft_consensus.cc:385] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.579172  1641 raft_consensus.cc:740] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.579571  1643 consensus_queue.cc:260] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.579989  1641 consensus_queue.cc:260] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.580118  1640 raft_consensus.cc:359] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.581005  1640 raft_consensus.cc:385] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.582592  1643 ts_tablet_manager.cc:1434] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:17:49.582991  1641 ts_tablet_manager.cc:1434] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:49.588981  1640 raft_consensus.cc:740] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.590054  1643 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.590150  1641 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.593027  1640 consensus_queue.cc:260] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.597491  1641 tablet_bootstrap.cc:654] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.600447  1640 ts_tablet_manager.cc:1434] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:17:49.601593  1640 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.604760  1643 tablet_bootstrap.cc:654] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.608202  1640 tablet_bootstrap.cc:654] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.620666  1640 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.621192  1640 ts_tablet_manager.cc:1403] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.005s
I20260709 12:17:49.623834  1643 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.624504  1643 ts_tablet_manager.cc:1403] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.012s
I20260709 12:17:49.624336  1640 raft_consensus.cc:359] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.625053  1640 raft_consensus.cc:385] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.625329  1640 raft_consensus.cc:740] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.626127  1641 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.626049  1640 consensus_queue.cc:260] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.626655  1641 ts_tablet_manager.cc:1403] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.010s
I20260709 12:17:49.627866  1643 raft_consensus.cc:359] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.628504  1640 ts_tablet_manager.cc:1434] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 12:17:49.628679  1643 raft_consensus.cc:385] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.633487  1643 raft_consensus.cc:740] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.630195  1641 raft_consensus.cc:359] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.633929  1641 raft_consensus.cc:385] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.634222  1641 raft_consensus.cc:740] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.634685  1618 raft_consensus.cc:493] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.634323  1643 consensus_queue.cc:260] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.635092  1641 consensus_queue.cc:260] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.636518  1643 ts_tablet_manager.cc:1434] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 12:17:49.636704  1623 raft_consensus.cc:493] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.637164  1623 raft_consensus.cc:515] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.637692  1643 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.639374  1623 leader_election.cc:290] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.640976  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1826dbb6bf8142c9a1dce2e574f75d6d" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.641323  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1826dbb6bf8142c9a1dce2e574f75d6d" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.641680  1348 raft_consensus.cc:2468] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 0.
I20260709 12:17:49.635210  1618 raft_consensus.cc:515] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.642912  1389 leader_election.cc:304] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
W20260709 12:17:49.643850  1389 leader_election.cc:343] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.644516  1643 tablet_bootstrap.cc:654] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.644750  1623 raft_consensus.cc:2804] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.645160  1623 raft_consensus.cc:493] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.645555  1623 raft_consensus.cc:3060] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.646195  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea47c15c0e64fd2868d4a3bf2466eed" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.646370  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea47c15c0e64fd2868d4a3bf2466eed" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.646889  1423 raft_consensus.cc:2468] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:49.646924  1497 raft_consensus.cc:2468] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:49.648684  1314 leader_election.cc:304] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:49.649600  1642 raft_consensus.cc:2804] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.649963  1642 raft_consensus.cc:493] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.650281  1642 raft_consensus.cc:3060] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.644769  1618 leader_election.cc:290] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.656499  1643 tablet_bootstrap.cc:492] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.657056  1643 ts_tablet_manager.cc:1403] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 12:17:49.659085  1623 raft_consensus.cc:515] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.660310  1643 raft_consensus.cc:359] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.661162  1643 raft_consensus.cc:385] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.661538  1643 raft_consensus.cc:740] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.662243  1640 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.662628  1643 consensus_queue.cc:260] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.664502  1641 ts_tablet_manager.cc:1434] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.003s
I20260709 12:17:49.665624  1641 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.665716  1643 ts_tablet_manager.cc:1434] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:17:49.668962  1643 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.671509  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1826dbb6bf8142c9a1dce2e574f75d6d" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:49.672235  1348 raft_consensus.cc:3060] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.673409  1640 tablet_bootstrap.cc:654] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.675624  1643 tablet_bootstrap.cc:654] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.680912  1348 raft_consensus.cc:2468] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:49.686177  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1826dbb6bf8142c9a1dce2e574f75d6d" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:49.686942  1497 raft_consensus.cc:3060] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.687511  1389 leader_election.cc:304] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:49.689077  1646 raft_consensus.cc:2804] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.693760  1654 raft_consensus.cc:493] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.694299  1654 raft_consensus.cc:515] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.694451  1647 raft_consensus.cc:493] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.695693  1647 raft_consensus.cc:515] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.696401  1654 leader_election.cc:290] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.700315  1497 raft_consensus.cc:2468] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:49.701195  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.701907  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.702847  1347 raft_consensus.cc:2468] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
W20260709 12:17:49.703146  1463 leader_election.cc:343] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.704051  1646 raft_consensus.cc:697] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:49.704710  1647 leader_election.cc:290] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.705174  1646 consensus_queue.cc:237] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.707639  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
W20260709 12:17:49.709895  1389 leader_election.cc:343] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.710232  1463 leader_election.cc:304] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:49.711901  1654 raft_consensus.cc:2804] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.712158  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.712415  1654 raft_consensus.cc:493] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.713522  1654 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.685757  1623 leader_election.cc:290] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
W20260709 12:17:49.719663  1389 leader_election.cc:343] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.720382  1389 leader_election.cc:304] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463; no voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.723241  1641 tablet_bootstrap.cc:654] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.724385  1670 raft_consensus.cc:2749] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:49.725085  1670 raft_consensus.cc:493] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.725548  1670 raft_consensus.cc:515] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.727883  1670 leader_election.cc:290] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.729660  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.725622  1642 raft_consensus.cc:515] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.730479  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
W20260709 12:17:49.731128  1389 leader_election.cc:343] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.731089  1497 raft_consensus.cc:2468] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 0.
I20260709 12:17:49.732292  1642 leader_election.cc:290] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.732587  1389 leader_election.cc:304] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc
I20260709 12:17:49.733224  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea47c15c0e64fd2868d4a3bf2466eed" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:49.733872  1423 raft_consensus.cc:3060] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.734009  1643 tablet_bootstrap.cc:492] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.734591  1643 ts_tablet_manager.cc:1403] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.066s	user 0.020s	sys 0.042s
I20260709 12:17:49.734937  1640 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.735466  1640 ts_tablet_manager.cc:1403] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.074s	user 0.026s	sys 0.011s
I20260709 12:17:49.737063  1643 raft_consensus.cc:359] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.737720  1643 raft_consensus.cc:385] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.738055  1643 raft_consensus.cc:740] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.738556  1670 raft_consensus.cc:2804] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.738453  1640 raft_consensus.cc:359] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.738855  1643 consensus_queue.cc:260] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.739418  1640 raft_consensus.cc:385] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.738960  1670 raft_consensus.cc:493] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.739941  1670 raft_consensus.cc:3060] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.740127  1641 tablet_bootstrap.cc:492] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.740557  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cea47c15c0e64fd2868d4a3bf2466eed" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:49.740899  1641 ts_tablet_manager.cc:1403] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.000s
I20260709 12:17:49.741137  1497 raft_consensus.cc:3060] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.744299  1641 raft_consensus.cc:359] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.745198  1641 raft_consensus.cc:385] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.745564  1641 raft_consensus.cc:740] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.746356  1641 consensus_queue.cc:260] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.747665  1670 raft_consensus.cc:515] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.748814  1641 ts_tablet_manager.cc:1434] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:49.749538  1640 raft_consensus.cc:740] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.749753  1497 raft_consensus.cc:2468] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:49.750191  1641 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.750466  1640 consensus_queue.cc:260] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.750974  1314 leader_election.cc:304] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:49.751835  1618 raft_consensus.cc:2804] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.752321  1618 raft_consensus.cc:697] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:49.753146  1618 consensus_queue.cc:237] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.741935  1643 ts_tablet_manager.cc:1434] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:17:49.757382  1643 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.757774  1641 tablet_bootstrap.cc:654] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.761008  1640 ts_tablet_manager.cc:1434] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:17:49.761729  1423 raft_consensus.cc:2468] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:49.757755  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:49.762893  1670 leader_election.cc:290] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.762239  1640 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.766580  1654 raft_consensus.cc:515] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.769040  1654 leader_election.cc:290] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.770147  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:49.770828  1496 raft_consensus.cc:3060] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.771178  1640 tablet_bootstrap.cc:654] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.771930  1643 tablet_bootstrap.cc:654] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.774703  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
W20260709 12:17:49.777792  1463 leader_election.cc:343] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.777927  1641 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.778005  1667 raft_consensus.cc:493] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.778627  1641 ts_tablet_manager.cc:1403] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.010s
I20260709 12:17:49.778739  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:49.779497  1347 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.778803  1667 raft_consensus.cc:515] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.782090  1641 raft_consensus.cc:359] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.782408  1667 leader_election.cc:290] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.782879  1641 raft_consensus.cc:385] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.783322  1641 raft_consensus.cc:740] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.784150  1641 consensus_queue.cc:260] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.784844  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.785759  1422 raft_consensus.cc:2468] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:49.786095  1654 raft_consensus.cc:493] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.786973  1641 ts_tablet_manager.cc:1434] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:17:49.786660  1654 raft_consensus.cc:515] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.788856  1654 leader_election.cc:290] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.788942  1495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.790174  1641 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: Bootstrap starting.
I20260709 12:17:49.791710  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:49.792905  1348 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:49.794329  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.795351  1422 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 0.
I20260709 12:17:49.794471  1463 leader_election.cc:304] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:49.796619  1654 raft_consensus.cc:2804] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.796984  1654 raft_consensus.cc:493] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.797991  1641 tablet_bootstrap.cc:654] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.797346  1654 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.788932  1314 leader_election.cc:304] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:49.802546  1667 raft_consensus.cc:2804] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.802975  1667 raft_consensus.cc:493] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.803364  1667 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.810981  1654 raft_consensus.cc:515] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
W20260709 12:17:49.811414  1314 leader_election.cc:343] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.800302  1243 catalog_manager.cc:5697] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:49.813948  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:49.814571  1348 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.814071  1246 catalog_manager.cc:5697] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:49.810981  1667 raft_consensus.cc:515] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
W20260709 12:17:49.819249  1389 leader_election.cc:343] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.821388  1496 raft_consensus.cc:2468] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 1.
I20260709 12:17:49.821992  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:49.822610  1422 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.822713  1654 leader_election.cc:290] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:49.822973  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:49.823633  1347 raft_consensus.cc:2468] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
W20260709 12:17:49.824589  1314 leader_election.cc:343] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:49.824854  1463 leader_election.cc:304] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:49.825567  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:49.826234  1654 raft_consensus.cc:2804] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.826064  1667 leader_election.cc:290] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.827108  1654 raft_consensus.cc:697] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:49.826924  1423 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.828037  1654 consensus_queue.cc:237] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.822633  1389 leader_election.cc:304] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc
I20260709 12:17:49.830399  1646 raft_consensus.cc:2804] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.830845  1646 raft_consensus.cc:697] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:49.831650  1646 consensus_queue.cc:237] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.836651  1423 raft_consensus.cc:2468] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:49.838455  1314 leader_election.cc:304] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.840142  1667 raft_consensus.cc:2804] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.840764  1667 raft_consensus.cc:697] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:49.841881  1667 consensus_queue.cc:237] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.842926  1348 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:49.850266  1422 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 1.
I20260709 12:17:49.851924  1463 leader_election.cc:304] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:49.853217  1658 raft_consensus.cc:2804] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.853856  1658 raft_consensus.cc:697] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:49.854915  1658 consensus_queue.cc:237] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.868168  1640 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.869627  1643 tablet_bootstrap.cc:492] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.870146  1643 ts_tablet_manager.cc:1403] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.113s	user 0.021s	sys 0.035s
I20260709 12:17:49.872054  1243 catalog_manager.cc:5697] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:49.872918  1643 raft_consensus.cc:359] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.873634  1643 raft_consensus.cc:385] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.873903  1643 raft_consensus.cc:740] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.874598  1643 consensus_queue.cc:260] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.876974  1643 ts_tablet_manager.cc:1434] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:17:49.877964  1643 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: Bootstrap starting.
I20260709 12:17:49.877216  1246 catalog_manager.cc:5697] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 1 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:49.883137  1641 tablet_bootstrap.cc:492] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: No bootstrap required, opened a new log
I20260709 12:17:49.883867  1641 ts_tablet_manager.cc:1403] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent bootstrapping tablet: real 0.094s	user 0.012s	sys 0.006s
I20260709 12:17:49.885347  1643 tablet_bootstrap.cc:654] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.887001  1641 raft_consensus.cc:359] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.887737  1641 raft_consensus.cc:385] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.887995  1641 raft_consensus.cc:740] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.888814  1641 consensus_queue.cc:260] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.891305  1641 ts_tablet_manager.cc:1434] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:17:49.868783  1640 ts_tablet_manager.cc:1403] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.107s	user 0.016s	sys 0.003s
I20260709 12:17:49.895620  1640 raft_consensus.cc:359] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.896147  1640 raft_consensus.cc:385] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.896363  1640 raft_consensus.cc:740] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.896994  1640 consensus_queue.cc:260] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.899417  1640 ts_tablet_manager.cc:1434] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:17:49.900323  1640 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: Bootstrap starting.
I20260709 12:17:49.900513  1643 tablet_bootstrap.cc:492] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: No bootstrap required, opened a new log
I20260709 12:17:49.901109  1643 ts_tablet_manager.cc:1403] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.015s
I20260709 12:17:49.904026  1643 raft_consensus.cc:359] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.904736  1643 raft_consensus.cc:385] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.905084  1643 raft_consensus.cc:740] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.899708  1244 catalog_manager.cc:5697] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:49.905864  1643 consensus_queue.cc:260] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.906908  1640 tablet_bootstrap.cc:654] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:49.907651  1643 ts_tablet_manager.cc:1434] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:17:49.911048  1246 catalog_manager.cc:5697] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 1 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:49.919875  1640 tablet_bootstrap.cc:492] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: No bootstrap required, opened a new log
I20260709 12:17:49.920526  1640 ts_tablet_manager.cc:1403] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.004s
I20260709 12:17:49.923386  1640 raft_consensus.cc:359] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.924140  1640 raft_consensus.cc:385] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:49.924479  1640 raft_consensus.cc:740] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Initialized, Role: FOLLOWER
I20260709 12:17:49.924655  1667 raft_consensus.cc:493] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.925207  1667 raft_consensus.cc:515] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.925338  1640 consensus_queue.cc:260] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.927354  1667 leader_election.cc:290] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.927559  1640 ts_tablet_manager.cc:1434] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:17:49.928886  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.929386  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.929555  1423 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:49.930083  1496 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:49.930873  1314 leader_election.cc:304] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:49.931684  1667 raft_consensus.cc:2804] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:49.932082  1667 raft_consensus.cc:493] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:49.932430  1667 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.938930  1667 raft_consensus.cc:515] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.941098  1667 leader_election.cc:290] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.941789  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:49.942076  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:49.942423  1423 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.942651  1496 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.944085  1667 raft_consensus.cc:493] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:49.944550  1667 raft_consensus.cc:515] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.946389  1667 leader_election.cc:290] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:49.947503  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:49.947921  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c544e2d317664736af713794ab7fb985" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:49.948630  1497 raft_consensus.cc:2393] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d972080851fd4e7784753234934d65bc in current term 1: Already voted for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in this term.
I20260709 12:17:49.949312  1423 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:49.949313  1496 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:49.949829  1314 leader_election.cc:304] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d972080851fd4e7784753234934d65bc; no voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:49.950691  1667 raft_consensus.cc:3060] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:49.951026  1314 leader_election.cc:304] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:49.951728  1642 raft_consensus.cc:2804] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:49.952193  1642 raft_consensus.cc:697] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:49.953007  1642 consensus_queue.cc:237] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:49.957512  1667 raft_consensus.cc:2749] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:49.961246  1246 catalog_manager.cc:5697] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.028426  1658 consensus_queue.cc:1048] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.040478  1654 consensus_queue.cc:1048] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.053021  1667 raft_consensus.cc:493] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.053602  1667 raft_consensus.cc:515] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.055312  1667 leader_election.cc:290] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.056146  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:50.056308  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" is_pre_election: true
I20260709 12:17:50.056694  1423 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:50.056994  1496 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 0.
I20260709 12:17:50.057732  1314 leader_election.cc:304] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:50.058494  1667 raft_consensus.cc:2804] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:50.058848  1667 raft_consensus.cc:493] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:50.059217  1667 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:50.064250  1667 raft_consensus.cc:515] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.065989  1667 leader_election.cc:290] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.066622  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:50.066919  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:50.067113  1423 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:50.067483  1496 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:50.073099  1423 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:50.074442  1314 leader_election.cc:304] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:50.074484  1496 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 1.
I20260709 12:17:50.076278  1667 raft_consensus.cc:2804] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:50.076800  1667 raft_consensus.cc:697] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:50.078436  1667 consensus_queue.cc:237] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.086784  1246 catalog_manager.cc:5697] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 0 to 1, leader changed from <none> to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 1 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.119943  1646 consensus_queue.cc:1048] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.131047  1623 consensus_queue.cc:1048] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.151962  1667 consensus_queue.cc:1048] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.152412  1658 raft_consensus.cc:493] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:50.152951  1658 raft_consensus.cc:515] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.155292  1658 leader_election.cc:290] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:50.156641  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" is_pre_election: true
I20260709 12:17:50.157415  1422 raft_consensus.cc:2393] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa3ca2b7e17e42c5becd0039a2ac0453 in current term 1: Already voted for candidate d972080851fd4e7784753234934d65bc in this term.
I20260709 12:17:50.157359  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d972080851fd4e7784753234934d65bc" is_pre_election: true
I20260709 12:17:50.159273  1463 leader_election.cc:304] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3ca2b7e17e42c5becd0039a2ac0453; no voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:50.161341  1658 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:50.163383  1642 consensus_queue.cc:1048] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.169375  1658 raft_consensus.cc:2749] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:50.183250  1496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
mode: GRACEFUL
new_leader_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:46642
I20260709 12:17:50.184034  1496 raft_consensus.cc:606] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Received request to transfer leadership to TS d972080851fd4e7784753234934d65bc
I20260709 12:17:50.195807  1496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
mode: GRACEFUL
new_leader_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:46642
I20260709 12:17:50.196313  1496 raft_consensus.cc:606] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Received request to transfer leadership to TS d972080851fd4e7784753234934d65bc
I20260709 12:17:50.202008  1348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:52284
I20260709 12:17:50.202569  1348 raft_consensus.cc:606] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:50.207893  1496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
mode: GRACEFUL
new_leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:46642
I20260709 12:17:50.208494  1496 raft_consensus.cc:606] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Received request to transfer leadership to TS f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:50.220283  1348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:52284
I20260709 12:17:50.220961  1348 raft_consensus.cc:606] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:50.239485  1658 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.244617  1658 consensus_queue.cc:1048] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.255786  1654 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.268867  1658 raft_consensus.cc:993] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: : Instructing follower d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:50.269440  1625 raft_consensus.cc:1081] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Signalling peer d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:50.275658  1348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3"
dest_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:52272
I20260709 12:17:50.276321  1348 raft_consensus.cc:493] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.276695  1348 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.279374  1625 raft_consensus.cc:993] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: : Instructing follower f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:50.279942  1625 raft_consensus.cc:1081] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Signalling peer f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:50.282744  1422 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:35526
I20260709 12:17:50.283325  1422 raft_consensus.cc:493] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.283684  1422 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.284698  1348 raft_consensus.cc:515] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.286388  1423 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:50.287948  1347 raft_consensus.cc:1240] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Rejecting Update request from peer fa3ca2b7e17e42c5becd0039a2ac0453 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:50.290149  1421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:50.294054  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5f2c8915f004e6499561957157d6fd3" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:50.294687  1496 raft_consensus.cc:3055] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.295009  1496 raft_consensus.cc:740] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:50.295455  1625 consensus_queue.cc:1059] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.295785  1348 leader_election.cc:290] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.296603  1654 consensus_queue.cc:1048] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.301638  1422 raft_consensus.cc:515] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.302098  1496 consensus_queue.cc:260] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.303854  1496 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.304177  1422 leader_election.cc:290] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.305509  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:50.306136  1348 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.306291  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:50.307091  1497 raft_consensus.cc:3055] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.307456  1497 raft_consensus.cc:740] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:50.308238  1497 consensus_queue.cc:260] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.309609  1497 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.313983  1348 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:50.314730  1496 raft_consensus.cc:2468] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
W20260709 12:17:50.315425  1625 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:17:50.315348  1389 leader_election.cc:304] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:50.316548  1670 raft_consensus.cc:2804] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.317178  1670 raft_consensus.cc:697] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:50.318094  1314 leader_election.cc:304] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: f3d8704396ba4bbcb6a0a52cf4a79463
I20260709 12:17:50.318114  1670 consensus_queue.cc:237] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.319434  1642 raft_consensus.cc:2804] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.319998  1642 raft_consensus.cc:697] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:50.320895  1642 consensus_queue.cc:237] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.321466  1497 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:50.328389  1244 catalog_manager.cc:5697] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 1 to 2, leader changed from fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195) to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 2 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.336791  1246 catalog_manager.cc:5697] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 1 to 2, leader changed from fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195) to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.346709  1642 consensus_queue.cc:1048] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.353680  1618 raft_consensus.cc:993] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: : Instructing follower f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:50.354183  1667 raft_consensus.cc:1081] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer f3d8704396ba4bbcb6a0a52cf4a79463 to start an election
I20260709 12:17:50.358515  1423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
 from {username='slave'} at 127.0.0.1:35536
I20260709 12:17:50.359073  1423 raft_consensus.cc:493] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.359411  1423 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.359292  1642 consensus_queue.cc:1048] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.369019  1423 raft_consensus.cc:515] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.370710  1423 leader_election.cc:290] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.371541  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:50.372236  1497 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd1a62cfa2748a1ba0ba2a91dce8286" candidate_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:50.372246  1348 raft_consensus.cc:3055] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.372756  1497 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.372768  1348 raft_consensus.cc:740] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:50.373737  1348 consensus_queue.cc:260] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.374728  1348 raft_consensus.cc:3060] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.378093  1497 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:50.379134  1389 leader_election.cc:304] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:50.379824  1670 raft_consensus.cc:2804] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.380277  1670 raft_consensus.cc:697] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Leader. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:50.380837  1348 raft_consensus.cc:2468] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3d8704396ba4bbcb6a0a52cf4a79463 in term 2.
I20260709 12:17:50.381098  1670 consensus_queue.cc:237] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.384147  1642 consensus_queue.cc:1048] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.390867  1244 catalog_manager.cc:5697] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194). New cstate: current_term: 2 leader_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.400298  1667 consensus_queue.cc:1048] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.409898  1670 consensus_queue.cc:1048] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.410351  1667 raft_consensus.cc:993] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: : Instructing follower fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:50.411017  1642 raft_consensus.cc:1081] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:50.412671  1496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:46626
I20260709 12:17:50.413261  1496 raft_consensus.cc:493] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.413635  1496 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.422041  1623 consensus_queue.cc:1048] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.421880  1496 raft_consensus.cc:515] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.427524  1347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:50.428140  1347 raft_consensus.cc:3055] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.428427  1347 raft_consensus.cc:740] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:50.429242  1347 consensus_queue.cc:260] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.430307  1347 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.435994  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df7058eb8b8e4c4e916d5c8368307ebe" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:50.436643  1423 raft_consensus.cc:3060] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.437567  1496 leader_election.cc:290] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:50.440948  1347 raft_consensus.cc:2468] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 2.
I20260709 12:17:50.443157  1463 leader_election.cc:304] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:50.443560  1423 raft_consensus.cc:2468] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 2.
I20260709 12:17:50.444025  1625 raft_consensus.cc:2804] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.444885  1625 raft_consensus.cc:697] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:50.445876  1625 consensus_queue.cc:237] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.453662  1246 catalog_manager.cc:5697] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:50.460979  1654 raft_consensus.cc:993] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: : Instructing follower d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:50.461598  1625 raft_consensus.cc:1081] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Signalling peer d972080851fd4e7784753234934d65bc to start an election
I20260709 12:17:50.463202  1347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c"
dest_uuid: "d972080851fd4e7784753234934d65bc"
 from {username='slave'} at 127.0.0.1:52272
I20260709 12:17:50.463665  1347 raft_consensus.cc:493] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:50.463977  1347 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.468581  1347 raft_consensus.cc:515] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.470026  1347 leader_election.cc:290] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Requested vote from peers f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391), fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251)
I20260709 12:17:50.470927  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:50.471202  1496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7494b6f417b94ffb8d1fb53b87c4f28c" candidate_uuid: "d972080851fd4e7784753234934d65bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
I20260709 12:17:50.471673  1422 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.471757  1496 raft_consensus.cc:3055] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:50.472157  1496 raft_consensus.cc:740] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:50.472999  1496 consensus_queue.cc:260] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.474299  1496 raft_consensus.cc:3060] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.479239  1422 raft_consensus.cc:2468] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:50.480713  1314 leader_election.cc:304] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, f3d8704396ba4bbcb6a0a52cf4a79463; no voters: 
I20260709 12:17:50.481586  1642 raft_consensus.cc:2804] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:50.481907  1496 raft_consensus.cc:2468] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d972080851fd4e7784753234934d65bc in term 2.
I20260709 12:17:50.482033  1642 raft_consensus.cc:697] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Leader. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:50.481976  1702 consensus_queue.cc:1048] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:50.483218  1642 consensus_queue.cc:237] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:50.493146  1246 catalog_manager.cc:5697] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc reported cstate change: term changed from 1 to 2, leader changed from fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195) to d972080851fd4e7784753234934d65bc (127.21.150.193). New cstate: current_term: 2 leader_uuid: "d972080851fd4e7784753234934d65bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:50.494666  1702 consensus_queue.cc:1048] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:50.750391  1347 raft_consensus.cc:1275] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.751544  1647 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.758898  1496 raft_consensus.cc:1275] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.760236  1623 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.771276  1347 raft_consensus.cc:1275] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.772487  1623 consensus_queue.cc:1048] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.777168  1496 raft_consensus.cc:1275] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer f3d8704396ba4bbcb6a0a52cf4a79463: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.779075  1646 consensus_queue.cc:1048] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.800809  1496 raft_consensus.cc:1275] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.802331  1667 consensus_queue.cc:1048] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.809830  1422 raft_consensus.cc:3060] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:50.817368  1422 raft_consensus.cc:1275] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.818774  1618 consensus_queue.cc:1048] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.838052  1347 raft_consensus.cc:1275] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.839205  1625 consensus_queue.cc:1048] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:50.850968  1423 raft_consensus.cc:1275] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:50.852594  1625 consensus_queue.cc:1048] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:51.056713  1496 raft_consensus.cc:1275] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:51.057924  1618 consensus_queue.cc:1048] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:51.069468  1423 raft_consensus.cc:1275] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer d972080851fd4e7784753234934d65bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:51.071688  1642 consensus_queue.cc:1048] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:53.225317  1246 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60764:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:17:53.246331  1246 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60764:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:17:53.268081 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:53.354943  1422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
mode: GRACEFUL
new_leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:52580
I20260709 12:17:53.355602  1422 raft_consensus.cc:606] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Received request to transfer leadership to TS fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:53.359351  1714 raft_consensus.cc:993] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: : Instructing follower fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:53.368579  1714 raft_consensus.cc:1081] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Signalling peer fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:53.373979  1493 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:46632
I20260709 12:17:53.374675  1493 raft_consensus.cc:493] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.375173  1493 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.412925  1765 tablet.cc:1724] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.423187  1766 tablet.cc:1724] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.433741  1763 tablet.cc:1724] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.438694  1770 tablet.cc:1724] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.423620  1493 raft_consensus.cc:515] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.474332  1492 raft_consensus.cc:1240] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Rejecting Update request from peer f3d8704396ba4bbcb6a0a52cf4a79463 for earlier term 2. Current term is 3. Ops: []
W20260709 12:17:53.458564  1749 log.cc:927] Time spent T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:17:53.489809  1775 tablet.cc:1724] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.491556  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
W20260709 12:17:53.492079  1523 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:cea47c15c0e64fd2868d4a3bf2466eed)
I20260709 12:17:53.492214  1422 raft_consensus.cc:3055] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.492591  1422 raft_consensus.cc:740] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:53.493592  1422 consensus_queue.cc:260] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.494750  1422 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:17:53.501657  1523 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:cea47c15c0e64fd2868d4a3bf2466eed)
W20260709 12:17:53.504745  1523 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:cea47c15c0e64fd2868d4a3bf2466eed)
I20260709 12:17:53.501183  1493 leader_election.cc:290] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:53.489887  1782 tablet.cc:1724] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.514998  1752 tablet.cc:1724] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.520116  1770 tablet.cc:1724] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.525976  1766 tablet.cc:1724] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.528847  1772 tablet.cc:1724] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.541896  1422 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:53.544143  1777 tablet.cc:1724] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.545954  1785 tablet.cc:1724] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.533195  1784 tablet.cc:1724] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.551687  1463 leader_election.cc:304] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:53.552904  1654 raft_consensus.cc:2804] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:53.553531  1654 raft_consensus.cc:697] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:53.561837  1654 consensus_queue.cc:237] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.514955  1765 tablet.cc:1724] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.563313  1763 tablet.cc:1724] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.573989  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1315022c36141fbb8a5defbda0e4168" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:53.575040  1348 raft_consensus.cc:3060] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.577035  1347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:47680
I20260709 12:17:53.579839  1347 raft_consensus.cc:606] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Received request to transfer leadership to TS fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:53.582083  1786 tablet.cc:1724] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.582736  1772 tablet.cc:1724] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.591810 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:17:53.593854  1752 tablet.cc:1724] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.596269 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:53.600996  1243 catalog_manager.cc:5697] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 2 to 3, leader changed from f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194) to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:53.603025  1243 catalog_manager.cc:5789] TS fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251) does not have the latest schema for tablet a1315022c36141fbb8a5defbda0e4168 (table soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]). Expected version 1 got 0
I20260709 12:17:53.611850  1775 tablet.cc:1724] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.613219  1423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496"
dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
mode: GRACEFUL
new_leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:52580
I20260709 12:17:53.613878  1423 raft_consensus.cc:606] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Received request to transfer leadership to TS fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:53.617092  1777 tablet.cc:1724] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.620012  1347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5"
dest_uuid: "d972080851fd4e7784753234934d65bc"
mode: GRACEFUL
new_leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:47680
I20260709 12:17:53.620693  1347 raft_consensus.cc:606] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Received request to transfer leadership to TS fa3ca2b7e17e42c5becd0039a2ac0453
I20260709 12:17:53.622599  1765 tablet.cc:1724] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.626251  1766 tablet.cc:1724] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.630755 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:53.633129 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:53.636729  1348 raft_consensus.cc:2468] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:53.642395  1770 tablet.cc:1724] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.643858  1772 tablet.cc:1724] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.649698  1777 tablet.cc:1724] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.663894  1244 catalog_manager.cc:5789] TS fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251) does not have the latest schema for tablet a1315022c36141fbb8a5defbda0e4168 (table soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]). Expected version 1 got 0
I20260709 12:17:53.665328  1772 tablet.cc:1724] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.702472  1246 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14] alter complete (version 1)
I20260709 12:17:53.825332  1711 raft_consensus.cc:993] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: : Instructing follower fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:53.825871  1711 raft_consensus.cc:1081] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Signalling peer fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:53.827396  1492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:46632
I20260709 12:17:53.827929  1492 raft_consensus.cc:493] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:53.828299  1492 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.834076  1492 raft_consensus.cc:515] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.836146  1492 leader_election.cc:290] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:53.836717  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:53.836993  1423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd0f4c451664f72a1f75274abfb2496" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:53.837258  1348 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.837575  1423 raft_consensus.cc:3055] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:53.837850  1423 raft_consensus.cc:740] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3d8704396ba4bbcb6a0a52cf4a79463, State: Running, Role: LEADER
I20260709 12:17:53.838505  1423 consensus_queue.cc:260] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.839210  1423 raft_consensus.cc:3060] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:53.844225  1348 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:53.844460  1423 raft_consensus.cc:2468] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:53.845559  1463 leader_election.cc:304] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d972080851fd4e7784753234934d65bc, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:53.846668  1654 raft_consensus.cc:2804] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:53.850492  1654 raft_consensus.cc:697] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:53.851300  1654 consensus_queue.cc:237] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:53.859117  1244 catalog_manager.cc:5697] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 2 to 3, leader changed from f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194) to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:53.970850  1423 raft_consensus.cc:1275] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:53.971998  1654 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:17:53.978554  1775 tablet.cc:1724] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:53.983669  1348 raft_consensus.cc:1275] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:17:53.987059  1767 consensus_queue.cc:1048] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:17:53.997867  1244 catalog_manager.cc:5789] TS fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195:39251) does not have the latest schema for tablet a1315022c36141fbb8a5defbda0e4168 (table soft_deleted_table [id=485f85aa99d1446fb407765042ab2d14]). Expected version 1 got 0
I20260709 12:17:54.018767  1775 tablet.cc:1719] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:17:54.020062  1765 tablet.cc:1719] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:17:54.024152  1766 tablet.cc:1719] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: Skipping requested alter to schema version 1, tablet already version 1
I20260709 12:17:54.031422  1712 raft_consensus.cc:993] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: : Instructing follower fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:54.032008  1739 raft_consensus.cc:1081] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Signalling peer fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:54.033638  1492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:46626
I20260709 12:17:54.034142  1492 raft_consensus.cc:493] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.034449  1492 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.039613  1492 raft_consensus.cc:515] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:54.041217  1492 leader_election.cc:290] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:54.042142  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:54.042332  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fdeefaeba5c24b7a96d2413978ccc4e5" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:54.042775  1348 raft_consensus.cc:3055] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:54.042847  1422 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.043169  1348 raft_consensus.cc:740] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:54.043982  1348 consensus_queue.cc:260] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:54.045171  1348 raft_consensus.cc:3060] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:54.050179  1422 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:54.051307  1463 leader_election.cc:304] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:54.051863  1348 raft_consensus.cc:2468] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 3.
I20260709 12:17:54.052168  1654 raft_consensus.cc:2804] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:54.052719  1654 raft_consensus.cc:697] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:54.053540  1654 consensus_queue.cc:237] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } }
I20260709 12:17:54.060426  1244 catalog_manager.cc:5697] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 2 to 3, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 3 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:54.109138  1712 raft_consensus.cc:993] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: : Instructing follower fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:54.109710  1739 raft_consensus.cc:1081] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Signalling peer fa3ca2b7e17e42c5becd0039a2ac0453 to start an election
I20260709 12:17:54.111683  1492 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b"
dest_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453"
 from {username='slave'} at 127.0.0.1:46626
I20260709 12:17:54.112365  1492 raft_consensus.cc:493] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:54.112762  1492 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.120119  1492 raft_consensus.cc:515] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:54.121915  1492 leader_election.cc:290] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 2 election: Requested vote from peers d972080851fd4e7784753234934d65bc (127.21.150.193:45907), f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391)
I20260709 12:17:54.122704  1348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "d972080851fd4e7784753234934d65bc"
I20260709 12:17:54.122969  1422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f62a468441f34f06bfca60fa9f63ac6b" candidate_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463"
I20260709 12:17:54.123382  1348 raft_consensus.cc:3055] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:54.123589  1422 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.123687  1348 raft_consensus.cc:740] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d972080851fd4e7784753234934d65bc, State: Running, Role: LEADER
I20260709 12:17:54.124421  1348 consensus_queue.cc:260] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:54.125334  1348 raft_consensus.cc:3060] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:54.128674  1422 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 2.
I20260709 12:17:54.129786  1463 leader_election.cc:304] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3d8704396ba4bbcb6a0a52cf4a79463, fa3ca2b7e17e42c5becd0039a2ac0453; no voters: 
I20260709 12:17:54.130146  1348 raft_consensus.cc:2468] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3ca2b7e17e42c5becd0039a2ac0453 in term 2.
I20260709 12:17:54.130632  1654 raft_consensus.cc:2804] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:54.131120  1654 raft_consensus.cc:697] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 LEADER]: Becoming Leader. State: Replica: fa3ca2b7e17e42c5becd0039a2ac0453, State: Running, Role: LEADER
I20260709 12:17:54.131942  1654 consensus_queue.cc:237] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } }
I20260709 12:17:54.139744  1244 catalog_manager.cc:5697] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 reported cstate change: term changed from 1 to 2, leader changed from d972080851fd4e7784753234934d65bc (127.21.150.193) to fa3ca2b7e17e42c5becd0039a2ac0453 (127.21.150.195). New cstate: current_term: 2 leader_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3ca2b7e17e42c5becd0039a2ac0453" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 39251 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:54.265285  1348 raft_consensus.cc:1275] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:54.266428  1654 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.275339  1422 raft_consensus.cc:1275] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:54.276829  1789 consensus_queue.cc:1048] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.479924  1422 raft_consensus.cc:1275] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:54.481168  1789 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.490470  1348 raft_consensus.cc:1275] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:54.491770  1654 consensus_queue.cc:1048] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.664798 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:54.696214  1348 raft_consensus.cc:1275] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:54.736042 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:54.736979 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:54.746490  1423 raft_consensus.cc:1275] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Refusing update from remote peer fa3ca2b7e17e42c5becd0039a2ac0453: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:17:54.754097  1767 consensus_queue.cc:1048] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d972080851fd4e7784753234934d65bc" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.755715  1767 consensus_queue.cc:1048] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3d8704396ba4bbcb6a0a52cf4a79463" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 36391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:54.800432 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:54.800941 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:17:55.643420  1369 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9740 after lost signal to thread 22110
W20260709 12:17:55.644559  1369 debug-util.cc:398] Leaking SignalData structure 0x7b08001d7a40 after lost signal to thread 1280
W20260709 12:17:55.645793  1369 debug-util.cc:398] Leaking SignalData structure 0x7b08001de280 after lost signal to thread 1372
W20260709 12:17:55.646772  1369 debug-util.cc:398] Leaking SignalData structure 0x7b080019ef00 after lost signal to thread 1447
W20260709 12:17:55.647812  1369 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4500 after lost signal to thread 1521
I20260709 12:17:55.802066 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:55.808034 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:55.808646 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:55.816488 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:55.817078 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:56.818078 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:56.823458 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:56.823917 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:56.828797 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:56.829198 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:57.830540 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:57.837510 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:57.838166 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:57.847915 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:57.848577 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:58.850067 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:58.855198 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:58.855842 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:58.861181 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:58.861763 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:59.862856 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:59.868876 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:59.869323 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:59.874071 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:59.874500 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:00.875576 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:00.882472 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:00.882905 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:00.887624 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:00.888084 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:01.889154 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:01.894940 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:01.895474 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:01.901007 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:01.901482 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:02.902513 22107 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:18:02.907605 22107 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:18:02.908069 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:02.913578 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:02.914296 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:03.981469 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
I20260709 12:18:04.026530 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:04.027329 22107 tablet_replica.cc:333] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.028465 22107 raft_consensus.cc:2243] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.029265 22107 raft_consensus.cc:2272] T c544e2d317664736af713794ab7fb985 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.032686 22107 tablet_replica.cc:333] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.033788 22107 raft_consensus.cc:2243] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.034351 22107 raft_consensus.cc:2272] T f62a468441f34f06bfca60fa9f63ac6b P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.037588 22107 tablet_replica.cc:333] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.038446 22107 raft_consensus.cc:2243] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.039122 22107 raft_consensus.cc:2272] T bbd1a62cfa2748a1ba0ba2a91dce8286 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.041710 22107 tablet_replica.cc:333] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.042335 22107 raft_consensus.cc:2243] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.043419 22107 raft_consensus.cc:2272] T cea47c15c0e64fd2868d4a3bf2466eed P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.046486 22107 tablet_replica.cc:333] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.047250 22107 raft_consensus.cc:2243] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.047966 22107 raft_consensus.cc:2272] T df7058eb8b8e4c4e916d5c8368307ebe P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:04.051503  1463 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 22 similar messages]
I20260709 12:18:04.051534 22107 tablet_replica.cc:333] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.052546 22107 raft_consensus.cc:2243] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.053314 22107 raft_consensus.cc:2272] T 1826dbb6bf8142c9a1dce2e574f75d6d P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:04.055003  1463 consensus_peers.cc:597] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.056424 22107 tablet_replica.cc:333] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.057140 22107 raft_consensus.cc:2243] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.057673 22107 raft_consensus.cc:2272] T 97ddf31261234c61853d015f42a162e9 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.060106 22107 tablet_replica.cc:333] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.060781 22107 raft_consensus.cc:2243] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.061357 22107 raft_consensus.cc:2272] T a1315022c36141fbb8a5defbda0e4168 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.063954 22107 tablet_replica.cc:333] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.064581 22107 raft_consensus.cc:2243] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.065366 22107 raft_consensus.cc:2272] T edc1782517cd4d2cbe842d3ea68e364c P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.068161 22107 tablet_replica.cc:333] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.068828 22107 raft_consensus.cc:2243] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.069837 22107 raft_consensus.cc:2272] T 577caaca97e64a5caa06f4d2d500db4c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.072281 22107 tablet_replica.cc:333] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.072911 22107 raft_consensus.cc:2243] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.073392 22107 raft_consensus.cc:2272] T 78b7ca1ca11049b590e84b9ff588270a P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.075902 22107 tablet_replica.cc:333] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.076514 22107 raft_consensus.cc:2243] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.077044 22107 raft_consensus.cc:2272] T 5b27df075eba4c6db0f45311133ff429 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.079602 22107 tablet_replica.cc:333] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.080188 22107 raft_consensus.cc:2243] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.081243 22107 raft_consensus.cc:2272] T 1c1a51e5426f4076a4f6ecc9887779c3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.083695 22107 tablet_replica.cc:333] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.084263 22107 raft_consensus.cc:2243] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.085284 22107 raft_consensus.cc:2272] T ade1be5869844d02a7e6cf1f195dc559 P d972080851fd4e7784753234934d65bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.087800 22107 tablet_replica.cc:333] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.088377 22107 raft_consensus.cc:2243] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.089282 22107 raft_consensus.cc:2272] T a5f2c8915f004e6499561957157d6fd3 P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.091656 22107 tablet_replica.cc:333] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.092187 22107 raft_consensus.cc:2243] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.092746 22107 raft_consensus.cc:2272] T bbd0f4c451664f72a1f75274abfb2496 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.094898 22107 tablet_replica.cc:333] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.095441 22107 raft_consensus.cc:2243] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.096365 22107 raft_consensus.cc:2272] T 7494b6f417b94ffb8d1fb53b87c4f28c P d972080851fd4e7784753234934d65bc [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.099316 22107 tablet_replica.cc:333] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc: stopping tablet replica
I20260709 12:18:04.099854 22107 raft_consensus.cc:2243] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.100435 22107 raft_consensus.cc:2272] T fdeefaeba5c24b7a96d2413978ccc4e5 P d972080851fd4e7784753234934d65bc [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.169317 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
W20260709 12:18:04.195288  1389 consensus_peers.cc:597] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:04.204285  1463 consensus_peers.cc:597] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:04.217839  1389 consensus_peers.cc:597] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.223819 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:18:04.233685  1463 consensus_peers.cc:597] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.258596 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:04.259689 22107 tablet_replica.cc:333] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.260545 22107 raft_consensus.cc:2243] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.261289 22107 raft_consensus.cc:2272] T a5f2c8915f004e6499561957157d6fd3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:04.262630  1463 consensus_peers.cc:597] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Couldn't send request to peer f3d8704396ba4bbcb6a0a52cf4a79463. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.264147 22107 tablet_replica.cc:333] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.264957 22107 raft_consensus.cc:2243] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.265616 22107 raft_consensus.cc:2272] T df7058eb8b8e4c4e916d5c8368307ebe P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.268026 22107 tablet_replica.cc:333] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.268755 22107 raft_consensus.cc:2243] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.269785 22107 raft_consensus.cc:2272] T 1826dbb6bf8142c9a1dce2e574f75d6d P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.272147 22107 tablet_replica.cc:333] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.273088 22107 raft_consensus.cc:2243] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.274113 22107 raft_consensus.cc:2272] T c544e2d317664736af713794ab7fb985 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.276794 22107 tablet_replica.cc:333] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.277521 22107 raft_consensus.cc:2243] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.278159 22107 raft_consensus.cc:2272] T f62a468441f34f06bfca60fa9f63ac6b P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.280750 22107 tablet_replica.cc:333] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.281404 22107 raft_consensus.cc:2243] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.281991 22107 raft_consensus.cc:2272] T bbd0f4c451664f72a1f75274abfb2496 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.284404 22107 tablet_replica.cc:333] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.285182 22107 raft_consensus.cc:2243] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.285738 22107 raft_consensus.cc:2272] T 78b7ca1ca11049b590e84b9ff588270a P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.289384 22107 tablet_replica.cc:333] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
W20260709 12:18:04.289572  1463 consensus_peers.cc:597] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Couldn't send request to peer f3d8704396ba4bbcb6a0a52cf4a79463. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.290266 22107 raft_consensus.cc:2243] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.290885 22107 raft_consensus.cc:2272] T 7494b6f417b94ffb8d1fb53b87c4f28c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.293296 22107 tablet_replica.cc:333] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.293915 22107 raft_consensus.cc:2243] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.294446 22107 raft_consensus.cc:2272] T fdeefaeba5c24b7a96d2413978ccc4e5 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.296835 22107 tablet_replica.cc:333] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.297469 22107 raft_consensus.cc:2243] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.298018 22107 raft_consensus.cc:2272] T 577caaca97e64a5caa06f4d2d500db4c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.300791 22107 tablet_replica.cc:333] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.301378 22107 raft_consensus.cc:2243] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.302402 22107 raft_consensus.cc:2272] T 5b27df075eba4c6db0f45311133ff429 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:04.304999  1463 consensus_peers.cc:597] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Couldn't send request to peer f3d8704396ba4bbcb6a0a52cf4a79463. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.305320 22107 tablet_replica.cc:333] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.306008 22107 raft_consensus.cc:2243] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.306535 22107 raft_consensus.cc:2272] T 1c1a51e5426f4076a4f6ecc9887779c3 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.308852 22107 tablet_replica.cc:333] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.309499 22107 raft_consensus.cc:2243] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.310020 22107 raft_consensus.cc:2272] T a1315022c36141fbb8a5defbda0e4168 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.312211 22107 tablet_replica.cc:333] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.312834 22107 raft_consensus.cc:2243] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.313932 22107 raft_consensus.cc:2272] T 97ddf31261234c61853d015f42a162e9 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.316197 22107 tablet_replica.cc:333] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.316803 22107 raft_consensus.cc:2243] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.317278 22107 raft_consensus.cc:2272] T ade1be5869844d02a7e6cf1f195dc559 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.319636 22107 tablet_replica.cc:333] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.320159 22107 raft_consensus.cc:2243] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.320662 22107 raft_consensus.cc:2272] T cea47c15c0e64fd2868d4a3bf2466eed P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.322886 22107 tablet_replica.cc:333] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.323395 22107 raft_consensus.cc:2243] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.324273 22107 raft_consensus.cc:2272] T bbd1a62cfa2748a1ba0ba2a91dce8286 P f3d8704396ba4bbcb6a0a52cf4a79463 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.326510 22107 tablet_replica.cc:333] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463: stopping tablet replica
I20260709 12:18:04.326980 22107 raft_consensus.cc:2243] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.327790 22107 raft_consensus.cc:2272] T edc1782517cd4d2cbe842d3ea68e364c P f3d8704396ba4bbcb6a0a52cf4a79463 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.372044 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
W20260709 12:18:04.373162  1463 consensus_peers.cc:597] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer d972080851fd4e7784753234934d65bc (127.21.150.193:45907): Couldn't send request to peer d972080851fd4e7784753234934d65bc. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:04.395601  1463 consensus_peers.cc:597] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 -> Peer f3d8704396ba4bbcb6a0a52cf4a79463 (127.21.150.194:36391): Couldn't send request to peer f3d8704396ba4bbcb6a0a52cf4a79463. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:36391: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:04.416313 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:18:04.446329 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:04.447153 22107 tablet_replica.cc:333] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.447911 22107 raft_consensus.cc:2243] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.448973 22107 raft_consensus.cc:2272] T f62a468441f34f06bfca60fa9f63ac6b P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.451504 22107 tablet_replica.cc:333] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.452113 22107 raft_consensus.cc:2243] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.453214 22107 raft_consensus.cc:2272] T df7058eb8b8e4c4e916d5c8368307ebe P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.455621 22107 tablet_replica.cc:333] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.456255 22107 raft_consensus.cc:2243] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.456748 22107 raft_consensus.cc:2272] T 1826dbb6bf8142c9a1dce2e574f75d6d P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.458981 22107 tablet_replica.cc:333] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.459605 22107 raft_consensus.cc:2243] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.460155 22107 raft_consensus.cc:2272] T c544e2d317664736af713794ab7fb985 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.462340 22107 tablet_replica.cc:333] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.462960 22107 raft_consensus.cc:2243] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.463462 22107 raft_consensus.cc:2272] T bbd1a62cfa2748a1ba0ba2a91dce8286 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.465551 22107 tablet_replica.cc:333] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.466111 22107 raft_consensus.cc:2243] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.466632 22107 raft_consensus.cc:2272] T cea47c15c0e64fd2868d4a3bf2466eed P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.468784 22107 tablet_replica.cc:333] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.469353 22107 raft_consensus.cc:2243] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.469878 22107 raft_consensus.cc:2272] T 577caaca97e64a5caa06f4d2d500db4c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.472541 22107 tablet_replica.cc:333] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.473130 22107 raft_consensus.cc:2243] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.474035 22107 raft_consensus.cc:2272] T fdeefaeba5c24b7a96d2413978ccc4e5 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.476572 22107 tablet_replica.cc:333] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.477042 22107 raft_consensus.cc:2243] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.477569 22107 raft_consensus.cc:2272] T 7494b6f417b94ffb8d1fb53b87c4f28c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.479540 22107 tablet_replica.cc:333] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.479995 22107 raft_consensus.cc:2243] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.480465 22107 raft_consensus.cc:2272] T 5b27df075eba4c6db0f45311133ff429 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.482440 22107 tablet_replica.cc:333] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.482960 22107 raft_consensus.cc:2243] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.483475 22107 raft_consensus.cc:2272] T a5f2c8915f004e6499561957157d6fd3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.485517 22107 tablet_replica.cc:333] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.486029 22107 raft_consensus.cc:2243] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.486912 22107 raft_consensus.cc:2272] T bbd0f4c451664f72a1f75274abfb2496 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.489059 22107 tablet_replica.cc:333] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.489656 22107 raft_consensus.cc:2243] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.490497 22107 raft_consensus.cc:2272] T 78b7ca1ca11049b590e84b9ff588270a P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.492478 22107 tablet_replica.cc:333] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.492992 22107 raft_consensus.cc:2243] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.493499 22107 raft_consensus.cc:2272] T 1c1a51e5426f4076a4f6ecc9887779c3 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.495741 22107 tablet_replica.cc:333] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.496253 22107 raft_consensus.cc:2243] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.496683 22107 raft_consensus.cc:2272] T edc1782517cd4d2cbe842d3ea68e364c P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.498803 22107 tablet_replica.cc:333] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.499322 22107 raft_consensus.cc:2243] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.499819 22107 raft_consensus.cc:2272] T ade1be5869844d02a7e6cf1f195dc559 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.502367 22107 tablet_replica.cc:333] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.502887 22107 raft_consensus.cc:2243] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.503798 22107 raft_consensus.cc:2272] T a1315022c36141fbb8a5defbda0e4168 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.506006 22107 tablet_replica.cc:333] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453: stopping tablet replica
I20260709 12:18:04.506518 22107 raft_consensus.cc:2243] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:04.507009 22107 raft_consensus.cc:2272] T 97ddf31261234c61853d015f42a162e9 P fa3ca2b7e17e42c5becd0039a2ac0453 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.549091 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:18:04.591023 22107 master.cc:562] Master@127.21.150.254:37815 shutting down...
I20260709 12:18:04.618629 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:04.619235 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:04.619575 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6af1e666f9854b0b8733b14e86ab5b11: stopping tablet replica
I20260709 12:18:04.644331 22107 master.cc:584] Master@127.21.150.254:37815 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (20639 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:18:04.710979 22107 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.150.254:38955
I20260709 12:18:04.712095 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:04.718317  1819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:04.718611  1820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:04.720093  1822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:04.721578 22107 server_base.cc:1061] running on GCE node
I20260709 12:18:04.723067 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:04.723420 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:04.723628 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599484723609 us; error 0 us; skew 500 ppm
I20260709 12:18:04.725821 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:04.728432 22107 webserver.cc:533] Webserver started at http://127.21.150.254:40825/ using document root <none> and password file <none>
I20260709 12:18:04.728962 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:04.729171 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:04.729518 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:04.730789 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/master-0-root/instance:
uuid: "d503b778e4fb47f9b81a48377edb6025"
format_stamp: "Formatted at 2026-07-09 12:18:04 on dist-test-slave-zctp"
I20260709 12:18:04.735432 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:18:04.739231  1827 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:04.740778 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:18:04.741132 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/master-0-root
uuid: "d503b778e4fb47f9b81a48377edb6025"
format_stamp: "Formatted at 2026-07-09 12:18:04 on dist-test-slave-zctp"
I20260709 12:18:04.741487 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:04.768119 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:04.769856 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:04.820251 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.254:38955
I20260709 12:18:04.820361  1878 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.254:38955 every 8 connection(s)
I20260709 12:18:04.824254  1879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:04.840080  1879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: Bootstrap starting.
I20260709 12:18:04.845664  1879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:04.850355  1879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: No bootstrap required, opened a new log
I20260709 12:18:04.852767  1879 raft_consensus.cc:359] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER }
I20260709 12:18:04.853216  1879 raft_consensus.cc:385] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:04.853521  1879 raft_consensus.cc:740] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d503b778e4fb47f9b81a48377edb6025, State: Initialized, Role: FOLLOWER
I20260709 12:18:04.854221  1879 consensus_queue.cc:260] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER }
I20260709 12:18:04.854777  1879 raft_consensus.cc:399] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:18:04.855026  1879 raft_consensus.cc:493] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:18:04.855314  1879 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:04.860558  1879 raft_consensus.cc:515] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER }
I20260709 12:18:04.861145  1879 leader_election.cc:304] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d503b778e4fb47f9b81a48377edb6025; no voters: 
I20260709 12:18:04.862511  1879 leader_election.cc:290] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:18:04.863322  1882 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:04.864691  1882 raft_consensus.cc:697] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 1 LEADER]: Becoming Leader. State: Replica: d503b778e4fb47f9b81a48377edb6025, State: Running, Role: LEADER
I20260709 12:18:04.865537  1882 consensus_queue.cc:237] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER }
I20260709 12:18:04.866418  1879 sys_catalog.cc:565] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:18:04.870980  1883 sys_catalog.cc:455] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d503b778e4fb47f9b81a48377edb6025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER } }
I20260709 12:18:04.871237  1884 sys_catalog.cc:455] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d503b778e4fb47f9b81a48377edb6025. Latest consensus state: current_term: 1 leader_uuid: "d503b778e4fb47f9b81a48377edb6025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d503b778e4fb47f9b81a48377edb6025" member_type: VOTER } }
I20260709 12:18:04.871985  1883 sys_catalog.cc:458] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:04.872068  1884 sys_catalog.cc:458] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [sys.catalog]: This master's current role is: LEADER
I20260709 12:18:04.876188  1888 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:18:04.881222  1888 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:18:04.886668 22107 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:18:04.892928  1888 catalog_manager.cc:1383] Generated new cluster ID: 1ad6d98ab8fd42cbaa1f73e03ac3f02c
I20260709 12:18:04.893245  1888 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:18:04.911226  1888 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:18:04.912652  1888 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:18:04.928818  1888 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: Generated new TSK 0
I20260709 12:18:04.929865  1888 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:18:04.958216 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:04.964529  1900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:04.965773  1901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:04.967137  1903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:04.968155 22107 server_base.cc:1061] running on GCE node
I20260709 12:18:04.969334 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:04.969645 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:04.969817 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599484969801 us; error 0 us; skew 500 ppm
I20260709 12:18:04.970386 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:04.972781 22107 webserver.cc:533] Webserver started at http://127.21.150.193:40901/ using document root <none> and password file <none>
I20260709 12:18:04.973276 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:04.973500 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:04.973784 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:04.974911 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-0-root/instance:
uuid: "019d73dc41d44c29920aa7460204384d"
format_stamp: "Formatted at 2026-07-09 12:18:04 on dist-test-slave-zctp"
I20260709 12:18:04.980587 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.002s
I20260709 12:18:04.984347  1908 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:04.985188 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:18:04.985553 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-0-root
uuid: "019d73dc41d44c29920aa7460204384d"
format_stamp: "Formatted at 2026-07-09 12:18:04 on dist-test-slave-zctp"
I20260709 12:18:04.985917 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:05.017863 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:05.019483 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:05.021162 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:05.024215 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:05.024554 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.024820 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:05.025017 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.071740 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.193:45695
I20260709 12:18:05.072221  1970 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.193:45695 every 8 connection(s)
I20260709 12:18:05.076748 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:05.085379  1975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:05.085726  1976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:05.090204 22107 server_base.cc:1061] running on GCE node
W20260709 12:18:05.090236  1978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:05.091627 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:18:05.091900 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:05.092412 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599485092392 us; error 0 us; skew 500 ppm
I20260709 12:18:05.093025 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:05.095569  1971 heartbeater.cc:344] Connected to a master server at 127.21.150.254:38955
I20260709 12:18:05.095949  1971 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:05.096346 22107 webserver.cc:533] Webserver started at http://127.21.150.194:46015/ using document root <none> and password file <none>
I20260709 12:18:05.096930 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:05.096839  1971 heartbeater.cc:507] Master 127.21.150.254:38955 requested a full tablet report, sending...
I20260709 12:18:05.097275 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:05.097637 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:05.099128 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-1-root/instance:
uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
format_stamp: "Formatted at 2026-07-09 12:18:05 on dist-test-slave-zctp"
I20260709 12:18:05.099866  1844 ts_manager.cc:194] Registered new tserver with Master: 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695)
I20260709 12:18:05.102614  1844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53160
I20260709 12:18:05.108126 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:18:05.113041  1983 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.113978 22107 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:18:05.114476 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-1-root
uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
format_stamp: "Formatted at 2026-07-09 12:18:05 on dist-test-slave-zctp"
I20260709 12:18:05.114914 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:05.129047 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:05.130422 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:05.131927 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:05.134158 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:05.134380 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.134624 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:05.134799 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.187088 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.194:35215
I20260709 12:18:05.187192  2045 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.194:35215 every 8 connection(s)
I20260709 12:18:05.193012 22107 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:18:05.201390  2050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:05.202248  2049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:18:05.203469  2052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:18:05.205469 22107 server_base.cc:1061] running on GCE node
I20260709 12:18:05.206620 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:18:05.206800  2046 heartbeater.cc:344] Connected to a master server at 127.21.150.254:38955
W20260709 12:18:05.206898 22107 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:18:05.207360  2046 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:05.207326 22107 hybrid_clock.cc:648] HybridClock initialized: now 1783599485207304 us; error 0 us; skew 500 ppm
I20260709 12:18:05.208209  2046 heartbeater.cc:507] Master 127.21.150.254:38955 requested a full tablet report, sending...
I20260709 12:18:05.208833 22107 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:18:05.211577  1844 ts_manager.cc:194] Registered new tserver with Master: 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:05.212913 22107 webserver.cc:533] Webserver started at http://127.21.150.195:39439/ using document root <none> and password file <none>
I20260709 12:18:05.213492  1844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53164
I20260709 12:18:05.213573 22107 fs_manager.cc:362] Metadata directory not provided
I20260709 12:18:05.214058 22107 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:18:05.214350 22107 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:18:05.215673 22107 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-2-root/instance:
uuid: "76776f5079244e5db5b08abda16c7aef"
format_stamp: "Formatted at 2026-07-09 12:18:05 on dist-test-slave-zctp"
I20260709 12:18:05.220544 22107 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:18:05.224541  2057 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.225586 22107 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:18:05.225958 22107 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-2-root
uuid: "76776f5079244e5db5b08abda16c7aef"
format_stamp: "Formatted at 2026-07-09 12:18:05 on dist-test-slave-zctp"
I20260709 12:18:05.226274 22107 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskf2cc6W/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599104329389-22107-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:18:05.255250 22107 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:18:05.256544 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:18:05.257954 22107 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:18:05.260280 22107 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:18:05.260531 22107 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.260788 22107 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:18:05.260967 22107 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:18:05.305871 22107 rpc_server.cc:307] RPC server started. Bound to: 127.21.150.195:41069
I20260709 12:18:05.305984  2119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.150.195:41069 every 8 connection(s)
I20260709 12:18:05.322046  2120 heartbeater.cc:344] Connected to a master server at 127.21.150.254:38955
I20260709 12:18:05.322517  2120 heartbeater.cc:461] Registering TS with master...
I20260709 12:18:05.323252  2120 heartbeater.cc:507] Master 127.21.150.254:38955 requested a full tablet report, sending...
I20260709 12:18:05.325203  1844 ts_manager.cc:194] Registered new tserver with Master: 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:05.325928 22107 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015877304s
I20260709 12:18:05.326762  1844 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53174
I20260709 12:18:05.355095  1844 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53180:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:05.358080  1844 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:05.435956  2085 tablet_service.cc:1511] Processing CreateTablet for tablet 89b3515fe56a41a0a60a492a7304bb9d (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:05.438716  2085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b3515fe56a41a0a60a492a7304bb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.440786  2084 tablet_service.cc:1511] Processing CreateTablet for tablet 130a5f34a2e2482bbc722ba9759d76a1 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:05.442296  2084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 130a5f34a2e2482bbc722ba9759d76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.457124  2083 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebabb8fe9df479bbac03224174cb2f9 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:05.458791  2083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebabb8fe9df479bbac03224174cb2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.463227  1936 tablet_service.cc:1511] Processing CreateTablet for tablet 89b3515fe56a41a0a60a492a7304bb9d (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:05.464918  1936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b3515fe56a41a0a60a492a7304bb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.465025  1934 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebabb8fe9df479bbac03224174cb2f9 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:05.466849  1935 tablet_service.cc:1511] Processing CreateTablet for tablet 130a5f34a2e2482bbc722ba9759d76a1 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:05.467343  1934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebabb8fe9df479bbac03224174cb2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.468196  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 130a5f34a2e2482bbc722ba9759d76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.475816  1933 tablet_service.cc:1511] Processing CreateTablet for tablet 7e680db9e7f1422aaf06275944c0db3a (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:05.477669  1933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e680db9e7f1422aaf06275944c0db3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.482515  2082 tablet_service.cc:1511] Processing CreateTablet for tablet 7e680db9e7f1422aaf06275944c0db3a (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:05.483973  2082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e680db9e7f1422aaf06275944c0db3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.486615  1932 tablet_service.cc:1511] Processing CreateTablet for tablet 99e2f17f371f4255bfece699c9a8a378 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:05.487917  1932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e2f17f371f4255bfece699c9a8a378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.490314  2007 tablet_service.cc:1511] Processing CreateTablet for tablet 99e2f17f371f4255bfece699c9a8a378 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:05.490568  2011 tablet_service.cc:1511] Processing CreateTablet for tablet 89b3515fe56a41a0a60a492a7304bb9d (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:05.491683  2007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e2f17f371f4255bfece699c9a8a378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.492780  2011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b3515fe56a41a0a60a492a7304bb9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.495227  2010 tablet_service.cc:1511] Processing CreateTablet for tablet 130a5f34a2e2482bbc722ba9759d76a1 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:05.498530  2009 tablet_service.cc:1511] Processing CreateTablet for tablet 5ebabb8fe9df479bbac03224174cb2f9 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:05.501865  2008 tablet_service.cc:1511] Processing CreateTablet for tablet 7e680db9e7f1422aaf06275944c0db3a (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:05.503981  2010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 130a5f34a2e2482bbc722ba9759d76a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.516898  2008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e680db9e7f1422aaf06275944c0db3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.523229  2009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ebabb8fe9df479bbac03224174cb2f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.560583  2084 tablet_service.cc:1511] Processing CreateTablet for tablet 99e2f17f371f4255bfece699c9a8a378 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:05.562500  2084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e2f17f371f4255bfece699c9a8a378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.573676  1931 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb08e64f1954232a431fd84f8cbef6e (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:05.575265  1931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb08e64f1954232a431fd84f8cbef6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.575019  2006 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb08e64f1954232a431fd84f8cbef6e (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:05.576422  2006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb08e64f1954232a431fd84f8cbef6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.614643  2084 tablet_service.cc:1511] Processing CreateTablet for tablet 0eb08e64f1954232a431fd84f8cbef6e (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:05.621984  2136 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:05.621682  1931 tablet_service.cc:1511] Processing CreateTablet for tablet f64a2d787148489595e36eba30e6d170 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:05.622996  2084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eb08e64f1954232a431fd84f8cbef6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.623178  1931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a2d787148489595e36eba30e6d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.628232  2135 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:05.647827  2137 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.671634  2136 tablet_bootstrap.cc:654] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.675113  2135 tablet_bootstrap.cc:654] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.687242  2007 tablet_service.cc:1511] Processing CreateTablet for tablet f64a2d787148489595e36eba30e6d170 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:05.689749  2007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a2d787148489595e36eba30e6d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.687775  2136 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:05.691910  2137 tablet_bootstrap.cc:654] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.692195  2136 ts_tablet_manager.cc:1403] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.006s
I20260709 12:18:05.693317  2082 tablet_service.cc:1511] Processing CreateTablet for tablet f64a2d787148489595e36eba30e6d170 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:05.694809  2082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a2d787148489595e36eba30e6d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.698668  2136 raft_consensus.cc:359] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.699851  2136 raft_consensus.cc:385] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.700306  2136 raft_consensus.cc:740] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.701354  2136 consensus_queue.cc:260] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.699452  2135 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:05.708845  2135 ts_tablet_manager.cc:1403] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.005s
I20260709 12:18:05.713167  2135 raft_consensus.cc:359] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.714157  2135 raft_consensus.cc:385] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.713716  2009 tablet_service.cc:1511] Processing CreateTablet for tablet e73bba6efab74c348ec9d818a055c266 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:05.714818  2135 raft_consensus.cc:740] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.715880  2009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73bba6efab74c348ec9d818a055c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.718721  2137 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:05.719307  2137 ts_tablet_manager.cc:1403] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.009s
I20260709 12:18:05.722257  2137 raft_consensus.cc:359] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.722987  2137 raft_consensus.cc:385] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.723274  2137 raft_consensus.cc:740] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.724138  2137 consensus_queue.cc:260] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.724963  1932 tablet_service.cc:1511] Processing CreateTablet for tablet e73bba6efab74c348ec9d818a055c266 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:05.726943  1932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73bba6efab74c348ec9d818a055c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.736712  2083 tablet_service.cc:1511] Processing CreateTablet for tablet e73bba6efab74c348ec9d818a055c266 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:05.738715  2083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e73bba6efab74c348ec9d818a055c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.731338  2135 consensus_queue.cc:260] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.747596  1935 tablet_service.cc:1511] Processing CreateTablet for tablet d7057c1494f5445d9ad8b3a1d9a06f29 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:05.748988  2137 ts_tablet_manager.cc:1434] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.029s	user 0.011s	sys 0.016s
I20260709 12:18:05.752763  2046 heartbeater.cc:499] Master 127.21.150.254:38955 was elected leader, sending a full tablet report...
I20260709 12:18:05.753185  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7057c1494f5445d9ad8b3a1d9a06f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.755596  2007 tablet_service.cc:1511] Processing CreateTablet for tablet d7057c1494f5445d9ad8b3a1d9a06f29 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:05.756927  2137 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.757100  2007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7057c1494f5445d9ad8b3a1d9a06f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.756902  2084 tablet_service.cc:1511] Processing CreateTablet for tablet d7057c1494f5445d9ad8b3a1d9a06f29 (DEFAULT_TABLE table=test-workload [id=a372281c66a04929b26f3d8cb7f0a298]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:05.760782  2084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7057c1494f5445d9ad8b3a1d9a06f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:05.775748  2120 heartbeater.cc:499] Master 127.21.150.254:38955 was elected leader, sending a full tablet report...
I20260709 12:18:05.778452  2137 tablet_bootstrap.cc:654] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.780764  2144 raft_consensus.cc:493] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.781749  1971 heartbeater.cc:499] Master 127.21.150.254:38955 was elected leader, sending a full tablet report...
I20260709 12:18:05.781369  2144 raft_consensus.cc:515] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.786542  2135 ts_tablet_manager.cc:1434] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.076s	user 0.011s	sys 0.005s
I20260709 12:18:05.799695  2144 leader_election.cc:290] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:05.802448  2135 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
W20260709 12:18:05.815948  2121 tablet.cc:2406] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:18:05.830098  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
W20260709 12:18:05.832000  1911 leader_election.cc:343] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:05.848174  2136 ts_tablet_manager.cc:1434] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.155s	user 0.025s	sys 0.026s
I20260709 12:18:05.849754  2136 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:05.860901  2135 tablet_bootstrap.cc:654] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.864243  2137 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:05.866097  2136 tablet_bootstrap.cc:654] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.866642  2137 ts_tablet_manager.cc:1403] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.035s
I20260709 12:18:05.870445  2137 raft_consensus.cc:359] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.871803  2137 raft_consensus.cc:385] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.872201  2137 raft_consensus.cc:740] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.873011  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:05.873395  2137 consensus_queue.cc:260] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
W20260709 12:18:05.874902  1910 leader_election.cc:343] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:05.875396  1910 leader_election.cc:304] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019d73dc41d44c29920aa7460204384d; no voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:05.876983  2141 raft_consensus.cc:2749] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:05.880348  2137 ts_tablet_manager.cc:1434] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
W20260709 12:18:05.880440  1892 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:05.881682  2137 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.882938  2135 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:05.883519  2135 ts_tablet_manager.cc:1403] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.008s
I20260709 12:18:05.885358  2136 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:05.886587  2136 ts_tablet_manager.cc:1403] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.011s
I20260709 12:18:05.886839  2135 raft_consensus.cc:359] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.887629  2135 raft_consensus.cc:385] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.887945  2135 raft_consensus.cc:740] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.889130  2137 tablet_bootstrap.cc:654] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.889051  2135 consensus_queue.cc:260] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.890242  2143 raft_consensus.cc:493] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:05.890661  2143 raft_consensus.cc:515] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.891077  2136 raft_consensus.cc:359] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.891904  2136 raft_consensus.cc:385] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.892271  2136 raft_consensus.cc:740] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.893399  2136 consensus_queue.cc:260] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.894981  2137 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:05.895550  2137 ts_tablet_manager.cc:1403] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 12:18:05.895612  2136 ts_tablet_manager.cc:1434] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 12:18:05.896677  2136 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:05.898793  2137 raft_consensus.cc:359] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.899554  2137 raft_consensus.cc:385] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.899856  2137 raft_consensus.cc:740] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.900645  2137 consensus_queue.cc:260] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.903873  2136 tablet_bootstrap.cc:654] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.907881  2137 ts_tablet_manager.cc:1434] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:18:05.909045  2137 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.915273  2135 ts_tablet_manager.cc:1434] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.008s
I20260709 12:18:05.917227  2135 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:05.924027  2143 leader_election.cc:290] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:05.925143  2135 tablet_bootstrap.cc:654] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.924600  2137 tablet_bootstrap.cc:654] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.930327  2136 tablet_bootstrap.cc:492] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:05.930930  2136 ts_tablet_manager.cc:1403] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.014s
I20260709 12:18:05.935009  2136 raft_consensus.cc:359] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.935966  2136 raft_consensus.cc:385] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.936313  2136 raft_consensus.cc:740] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.937283  2136 consensus_queue.cc:260] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:05.938862  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:05.940161  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
W20260709 12:18:05.940662  2061 leader_election.cc:343] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:05.940089  2137 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
W20260709 12:18:05.941867  2060 leader_election.cc:343] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:05.942695  2136 ts_tablet_manager.cc:1434] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:18:05.942669  2060 leader_election.cc:304] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76776f5079244e5db5b08abda16c7aef; no voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:05.943691  2143 raft_consensus.cc:2749] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:05.944057  2136 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:05.944331  2137 ts_tablet_manager.cc:1403] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.007s
I20260709 12:18:05.948000  2137 raft_consensus.cc:359] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.948865  2137 raft_consensus.cc:385] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.949182  2137 raft_consensus.cc:740] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.950152  2137 consensus_queue.cc:260] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.951908  2136 tablet_bootstrap.cc:654] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.957252  2135 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:05.957216  2137 ts_tablet_manager.cc:1434] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:18:05.958176  2135 ts_tablet_manager.cc:1403] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.010s
I20260709 12:18:05.959108  2137 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.962343  2135 raft_consensus.cc:359] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.963351  2135 raft_consensus.cc:385] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.963728  2135 raft_consensus.cc:740] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.964895  2135 consensus_queue.cc:260] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.966001  2137 tablet_bootstrap.cc:654] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.970141  2135 ts_tablet_manager.cc:1434] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 12:18:05.970916  2136 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:05.971290  2137 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:05.971441  2135 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:05.971477  2136 ts_tablet_manager.cc:1403] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.010s
I20260709 12:18:05.971836  2137 ts_tablet_manager.cc:1403] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:18:05.975072  2137 raft_consensus.cc:359] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.975294  2136 raft_consensus.cc:359] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.975878  2137 raft_consensus.cc:385] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.976097  2136 raft_consensus.cc:385] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:05.976219  2137 raft_consensus.cc:740] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.976550  2136 raft_consensus.cc:740] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:05.977187  2137 consensus_queue.cc:260] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.977397  2136 consensus_queue.cc:260] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:05.979645  2135 tablet_bootstrap.cc:654] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.979946  2136 ts_tablet_manager.cc:1434] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:18:05.982100  2136 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:05.983749  2137 ts_tablet_manager.cc:1434] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:18:05.984908  2137 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:05.988906  2136 tablet_bootstrap.cc:654] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:05.992575  2137 tablet_bootstrap.cc:654] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.002811  2136 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:06.003571  2136 ts_tablet_manager.cc:1403] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.012s
I20260709 12:18:06.006299  2135 tablet_bootstrap.cc:492] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.006930  2137 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:06.007000  2135 ts_tablet_manager.cc:1403] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.020s
I20260709 12:18:06.007128  2136 raft_consensus.cc:359] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.007853  2137 ts_tablet_manager.cc:1403] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 12:18:06.008277  2136 raft_consensus.cc:385] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.008661  2136 raft_consensus.cc:740] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.009720  2136 consensus_queue.cc:260] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.010797  2135 raft_consensus.cc:359] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.011544  2135 raft_consensus.cc:385] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.011822  2135 raft_consensus.cc:740] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.012439  2136 ts_tablet_manager.cc:1434] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:18:06.011958  2137 raft_consensus.cc:359] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.012904  2137 raft_consensus.cc:385] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.013410  2137 raft_consensus.cc:740] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.012847  2135 consensus_queue.cc:260] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.013710  2136 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:06.014417  2137 consensus_queue.cc:260] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.015784  2135 ts_tablet_manager.cc:1434] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:18:06.017145  2135 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:06.019899  2137 ts_tablet_manager.cc:1434] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:18:06.021262  2137 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:06.021965  2136 tablet_bootstrap.cc:654] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.025158  2135 tablet_bootstrap.cc:654] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.029160  2137 tablet_bootstrap.cc:654] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.032688  2141 raft_consensus.cc:493] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.033401  2141 raft_consensus.cc:515] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.036038  2141 leader_election.cc:290] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.037199  2135 tablet_bootstrap.cc:492] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.037781  2135 ts_tablet_manager.cc:1403] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.011s
I20260709 12:18:06.038100  2143 raft_consensus.cc:493] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.038797  2143 raft_consensus.cc:515] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.041590  2143 leader_election.cc:290] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695)
I20260709 12:18:06.041185  2135 raft_consensus.cc:359] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.042048  2137 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:06.042137  2135 raft_consensus.cc:385] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.037786  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e680db9e7f1422aaf06275944c0db3a" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.042775  2135 raft_consensus.cc:740] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.042955  2137 ts_tablet_manager.cc:1403] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.003s
I20260709 12:18:06.043238  2021 raft_consensus.cc:2468] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:06.043748  2135 consensus_queue.cc:260] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.045049  1911 leader_election.cc:304] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.046428  2135 ts_tablet_manager.cc:1434] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:18:06.046701  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.047309  1946 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:06.047519  2135 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:06.047369  2137 raft_consensus.cc:359] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.048715  2137 raft_consensus.cc:385] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.049093  2137 raft_consensus.cc:740] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.049638  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e680db9e7f1422aaf06275944c0db3a" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.050107  2137 consensus_queue.cc:260] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.051926  2061 leader_election.cc:304] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.053017  2143 raft_consensus.cc:2804] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.053453  2143 raft_consensus.cc:493] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.053843  2143 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.053866  2137 ts_tablet_manager.cc:1434] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 12:18:06.055828  2135 tablet_bootstrap.cc:654] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.055869  2137 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
W20260709 12:18:06.058204  1910 leader_election.cc:343] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.060101  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.060900  2021 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:06.065842  2143 raft_consensus.cc:515] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.069504  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.070305  2021 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.074293  2141 raft_consensus.cc:2804] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.074846  2141 raft_consensus.cc:493] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.075232  2141 raft_consensus.cc:3060] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.076895  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.077669  1946 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.084666  2143 leader_election.cc:290] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695)
I20260709 12:18:06.088907  2137 tablet_bootstrap.cc:654] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.090732  1946 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:06.092056  2141 raft_consensus.cc:515] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.094506  2021 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:06.096354  2061 leader_election.cc:304] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.096407  2141 leader_election.cc:290] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.098673  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e680db9e7f1422aaf06275944c0db3a" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.099851  2135 tablet_bootstrap.cc:492] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.100028  2143 raft_consensus.cc:2804] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.100476  2135 ts_tablet_manager.cc:1403] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.005s
I20260709 12:18:06.101466  2137 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:06.099880  2020 raft_consensus.cc:3060] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.101514  2143 raft_consensus.cc:697] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:06.102151  2137 ts_tablet_manager.cc:1403] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.004s
I20260709 12:18:06.103034  2143 consensus_queue.cc:237] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.104310  2135 raft_consensus.cc:359] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.101375  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e680db9e7f1422aaf06275944c0db3a" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.105199  2135 raft_consensus.cc:385] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.104966  2137 raft_consensus.cc:359] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.105669  2135 raft_consensus.cc:740] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.105805  2137 raft_consensus.cc:385] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.106205  2137 raft_consensus.cc:740] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.106938  2137 consensus_queue.cc:260] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.109304  2137 ts_tablet_manager.cc:1434] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:18:06.110049  2142 raft_consensus.cc:493] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.110489  2137 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:06.110863  2136 tablet_bootstrap.cc:492] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:06.111673  2136 ts_tablet_manager.cc:1403] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.098s	user 0.036s	sys 0.043s
I20260709 12:18:06.110829  2142 raft_consensus.cc:515] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.114876  2142 leader_election.cc:290] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.114843  2136 raft_consensus.cc:359] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.115906  2136 raft_consensus.cc:385] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.116276  2136 raft_consensus.cc:740] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.116858  2137 tablet_bootstrap.cc:654] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.117214  2136 consensus_queue.cc:260] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.120208  2136 ts_tablet_manager.cc:1434] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:18:06.121476  2136 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:06.129914  2136 tablet_bootstrap.cc:654] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.131317  2137 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:06.131920  2137 ts_tablet_manager.cc:1403] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 12:18:06.135600  2137 raft_consensus.cc:359] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.106621  2135 consensus_queue.cc:260] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.136574  2137 raft_consensus.cc:385] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.136888  2095 raft_consensus.cc:3060] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.136991  2137 raft_consensus.cc:740] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.138459  2137 consensus_queue.cc:260] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.139153  2020 raft_consensus.cc:2468] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.140552  1911 leader_election.cc:304] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.141467  2141 raft_consensus.cc:2804] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.141839  2137 ts_tablet_manager.cc:1434] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:18:06.141947  2141 raft_consensus.cc:697] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.143163  2141 consensus_queue.cc:237] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.145187  2095 raft_consensus.cc:2468] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.145530  2136 tablet_bootstrap.cc:492] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:06.146790  2136 ts_tablet_manager.cc:1403] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.007s
I20260709 12:18:06.151561  2136 raft_consensus.cc:359] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.152832  2136 raft_consensus.cc:385] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.153301  2136 raft_consensus.cc:740] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.154492  2136 consensus_queue.cc:260] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.157274  2135 ts_tablet_manager.cc:1434] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.056s	user 0.012s	sys 0.002s
I20260709 12:18:06.157346  2136 ts_tablet_manager.cc:1434] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:18:06.158999  2136 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:06.165266  2142 raft_consensus.cc:493] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.165314  2135 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:06.165947  2142 raft_consensus.cc:515] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.168289  1844 catalog_manager.cc:5697] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 0 to 1, leader changed from <none> to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 1 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.169059  2142 leader_election.cc:290] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.172832  2136 tablet_bootstrap.cc:654] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.176610  2135 tablet_bootstrap.cc:654] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.178465  2141 raft_consensus.cc:493] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.179121  2141 raft_consensus.cc:515] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.182976  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.183733  2020 raft_consensus.cc:2468] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:06.185006  1911 leader_election.cc:304] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.186040  2144 raft_consensus.cc:2804] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.186501  2144 raft_consensus.cc:493] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.186885  2144 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.179020  1843 catalog_manager.cc:5697] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.199065  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.199290  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.200320  1945 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.201611  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.202045  1987 leader_election.cc:304] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.202426  2095 raft_consensus.cc:2468] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.203155  2142 raft_consensus.cc:2804] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.203414  2136 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:06.203573  2142 raft_consensus.cc:493] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.204160  2142 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.204170  2136 ts_tablet_manager.cc:1403] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.012s
I20260709 12:18:06.206607  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.207412  2095 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.208499  2136 raft_consensus.cc:359] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.196982  2144 raft_consensus.cc:515] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.209569  2136 raft_consensus.cc:385] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.210461  2136 raft_consensus.cc:740] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.211431  1946 raft_consensus.cc:2393] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6428ce197bfd4557b67da6a3eca8a0ca in current term 1: Already voted for candidate 019d73dc41d44c29920aa7460204384d in this term.
I20260709 12:18:06.211534  2136 consensus_queue.cc:260] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.209227  1985 leader_election.cc:304] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.213510  2144 leader_election.cc:290] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.212332  2141 leader_election.cc:290] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.215194  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.217964  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.218506  2094 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.224330  2135 tablet_bootstrap.cc:492] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.225723  2135 ts_tablet_manager.cc:1403] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.067s	user 0.026s	sys 0.025s
I20260709 12:18:06.225759  2177 raft_consensus.cc:2804] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.226449  2177 raft_consensus.cc:493] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.218485  2144 raft_consensus.cc:493] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.232522  2177 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.232702  2144 raft_consensus.cc:515] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.235054  2144 leader_election.cc:290] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.227499  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.229257  2135 raft_consensus.cc:359] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.231541  2136 ts_tablet_manager.cc:1434] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.001s
I20260709 12:18:06.235749  2135 raft_consensus.cc:385] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.236073  2135 raft_consensus.cc:740] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.236331  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.236573  2136 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:06.237036  2095 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:06.236856  2135 consensus_queue.cc:260] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.238284  1910 leader_election.cc:304] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.232143  2176 raft_consensus.cc:493] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.239195  2135 ts_tablet_manager.cc:1434] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:18:06.239243  2144 raft_consensus.cc:2804] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.239830  2144 raft_consensus.cc:493] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.240197  2144 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.240386  2135 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:06.243691  2136 tablet_bootstrap.cc:654] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.246556  2094 raft_consensus.cc:2468] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.248144  1910 leader_election.cc:304] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.249303  2141 raft_consensus.cc:2804] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.249897  2141 raft_consensus.cc:697] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.250751  2135 tablet_bootstrap.cc:654] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.243201  2176 raft_consensus.cc:515] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.251013  2141 consensus_queue.cc:237] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.254876  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
W20260709 12:18:06.256845  1910 leader_election.cc:343] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.257601  2176 leader_election.cc:290] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.258847  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.259784  2021 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:06.262599  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.263473  2021 raft_consensus.cc:2468] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:06.266260  2177 raft_consensus.cc:515] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.266678  1911 leader_election.cc:304] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.233338  2142 raft_consensus.cc:515] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.267829  2020 raft_consensus.cc:2393] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 019d73dc41d44c29920aa7460204384d in current term 1: Already voted for candidate 6428ce197bfd4557b67da6a3eca8a0ca in this term.
I20260709 12:18:06.269309  2176 raft_consensus.cc:2804] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.269830  2176 raft_consensus.cc:493] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.270251  2176 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.270474  2177 leader_election.cc:290] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.271835  2136 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:06.271867  2177 raft_consensus.cc:493] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.272845  2136 ts_tablet_manager.cc:1403] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.017s
I20260709 12:18:06.272784  2177 raft_consensus.cc:515] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.274915  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.276176  2095 raft_consensus.cc:2393] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6428ce197bfd4557b67da6a3eca8a0ca in current term 1: Already voted for candidate 019d73dc41d44c29920aa7460204384d in this term.
I20260709 12:18:06.276576  2136 raft_consensus.cc:359] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.277410  2136 raft_consensus.cc:385] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.277799  2136 raft_consensus.cc:740] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.278652  2136 consensus_queue.cc:260] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.280005  2177 leader_election.cc:290] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.281234  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.282019  2136 ts_tablet_manager.cc:1434] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
W20260709 12:18:06.282829  1985 leader_election.cc:343] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.283293  2135 tablet_bootstrap.cc:492] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.283727  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.283828  2135 ts_tablet_manager.cc:1403] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.000s
I20260709 12:18:06.284789  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.285183  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.285566  1945 raft_consensus.cc:2468] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.285907  1946 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.287094  2142 leader_election.cc:290] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.287377  2135 raft_consensus.cc:359] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.287833  1987 leader_election.cc:304] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.288290  2135 raft_consensus.cc:385] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.288717  2135 raft_consensus.cc:740] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.288827  2177 raft_consensus.cc:2749] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:18:06.289285  1987 leader_election.cc:304] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.290212  2177 raft_consensus.cc:2804] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.289872  2135 consensus_queue.cc:260] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.290701  2177 raft_consensus.cc:493] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.291144  2177 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.292982  2176 raft_consensus.cc:515] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.294176  2143 raft_consensus.cc:493] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.294766  2143 raft_consensus.cc:515] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.297129  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.298431  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.299070  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.299171  1945 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:06.299854  2020 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:06.300094  2176 leader_election.cc:290] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.299161  2144 raft_consensus.cc:515] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.300740  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.301386  2061 leader_election.cc:304] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.301611  1946 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.301419  2094 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.302631  2151 raft_consensus.cc:2804] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.303067  2151 raft_consensus.cc:493] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.303429  2151 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.305287  2144 leader_election.cc:290] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
W20260709 12:18:06.299202  1910 leader_election.cc:343] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:06.306651  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.297132  2143 leader_election.cc:290] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:06.307472  2020 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.308821  2135 ts_tablet_manager.cc:1434] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:18:06.309954  2135 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:06.315164  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.316215  2095 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.317306  2094 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.323110  2020 raft_consensus.cc:2468] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.323342  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.323939  2021 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.324080  1987 leader_election.cc:304] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.325251  2142 raft_consensus.cc:2804] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.325929  2142 raft_consensus.cc:697] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.326025  2177 raft_consensus.cc:515] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.326949  2142 consensus_queue.cc:237] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.330507  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.331660  1946 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.333688  2021 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.336010  2095 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:06.330377  1911 leader_election.cc:304] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.341650  2141 raft_consensus.cc:2804] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.342190  2141 raft_consensus.cc:697] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.339596  2151 raft_consensus.cc:515] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.343685  2141 consensus_queue.cc:237] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.346599  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.347376  1945 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.342199  1911 leader_election.cc:304] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.351480  2144 raft_consensus.cc:2804] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.352196  2144 raft_consensus.cc:697] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.353318  1946 raft_consensus.cc:2468] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.353397  2144 consensus_queue.cc:237] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.356635  1945 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:06.328392  2177 leader_election.cc:290] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.315778  2135 tablet_bootstrap.cc:654] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:06.359143  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.360281  2021 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.340270  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.359208  2151 leader_election.cc:290] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:06.363545  1987 leader_election.cc:304] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.365947  2177 raft_consensus.cc:2804] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.366428  2177 raft_consensus.cc:697] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.367828  2177 consensus_queue.cc:237] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.364491  2094 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.372272  2061 leader_election.cc:304] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.373874  1843 catalog_manager.cc:5697] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.361318  1844 catalog_manager.cc:5697] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.376583  2021 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:06.379282  2094 raft_consensus.cc:2468] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.383467  2151 raft_consensus.cc:2804] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.384495  2151 raft_consensus.cc:697] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:06.385589  2151 consensus_queue.cc:237] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.392254  2177 raft_consensus.cc:493] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.392828  2177 raft_consensus.cc:515] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.394209  1843 catalog_manager.cc:5697] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.395923  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.396847  2177 leader_election.cc:290] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.397653  2135 tablet_bootstrap.cc:492] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:06.397709  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:06.397356  1841 catalog_manager.cc:5697] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.398314  2135 ts_tablet_manager.cc:1403] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.089s	user 0.008s	sys 0.008s
I20260709 12:18:06.398465  2094 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.399664  1985 leader_election.cc:304] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.399255  1841 catalog_manager.cc:5697] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.400550  2177 raft_consensus.cc:2804] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:06.400975  2177 raft_consensus.cc:493] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:06.401520  2177 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.401518  2135 raft_consensus.cc:359] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.402609  2135 raft_consensus.cc:385] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:06.402954  2135 raft_consensus.cc:740] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:06.404129  2135 consensus_queue.cc:260] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.396847  1945 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:06.408246  2135 ts_tablet_manager.cc:1434] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:18:06.409943  2177 raft_consensus.cc:515] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.412330  2177 leader_election.cc:290] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.413179  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.413534  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.413924  1945 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.414134  2094 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.417513  1843 catalog_manager.cc:5697] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 0 to 1, leader changed from <none> to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 1 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.421979  1945 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.422129  2094 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:06.423133  1987 leader_election.cc:304] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.423882  2177 raft_consensus.cc:2804] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:06.424367  2177 raft_consensus.cc:697] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.425158  2177 consensus_queue.cc:237] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.434844  1843 catalog_manager.cc:5697] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.560662  2151 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.576054  2143 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.579852  2094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
mode: GRACEFUL
new_leader_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:18:06.580865  2094 raft_consensus.cc:606] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Received request to transfer leadership to TS 019d73dc41d44c29920aa7460204384d
I20260709 12:18:06.600395  2195 raft_consensus.cc:993] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: : Instructing follower 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:06.600946  2151 raft_consensus.cc:1081] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Signalling peer 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:06.602970  1945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:53978
I20260709 12:18:06.604178  1945 raft_consensus.cc:493] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:06.604614  1945 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.607177  2021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
mode: GRACEFUL
new_leader_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:49178
I20260709 12:18:06.607991  2021 raft_consensus.cc:606] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Received request to transfer leadership to TS 019d73dc41d44c29920aa7460204384d
I20260709 12:18:06.613096  1945 raft_consensus.cc:515] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.615366  1945 leader_election.cc:290] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Requested vote from peers 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:06.616629  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.616688  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.617627  2094 raft_consensus.cc:3055] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.617674  2020 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.618254  2094 raft_consensus.cc:740] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:06.618317  2021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
mode: GRACEFUL
new_leader_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:49178
I20260709 12:18:06.619029  2021 raft_consensus.cc:606] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Received request to transfer leadership to TS 019d73dc41d44c29920aa7460204384d
I20260709 12:18:06.619542  2094 consensus_queue.cc:260] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.621341  2094 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.627830  2020 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:06.629331  1911 leader_election.cc:304] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:06.629911  2094 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:06.630169  2144 raft_consensus.cc:2804] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.630790  2144 raft_consensus.cc:697] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.631574  2144 consensus_queue.cc:237] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:06.633217  2141 consensus_queue.cc:1048] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.639408  1841 catalog_manager.cc:5697] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 1 to 2, leader changed from 76776f5079244e5db5b08abda16c7aef (127.21.150.195) to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 2 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:06.643102  1945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:53986
I20260709 12:18:06.644040  1945 raft_consensus.cc:606] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:06.654621  2176 consensus_queue.cc:1048] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.656528  1945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:53986
I20260709 12:18:06.657276  1945 raft_consensus.cc:606] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:06.667641  1945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f64a2d787148489595e36eba30e6d170"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:53986
I20260709 12:18:06.668248  1945 raft_consensus.cc:606] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:06.674546  2094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:18:06.675241  2094 raft_consensus.cc:606] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:06.681483  2020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
mode: GRACEFUL
new_leader_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:49178
I20260709 12:18:06.681975  2020 raft_consensus.cc:606] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Received request to transfer leadership to TS 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.733534  2144 consensus_queue.cc:1048] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.740468  2178 raft_consensus.cc:993] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.740983  2176 raft_consensus.cc:1081] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.742828  2021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f64a2d787148489595e36eba30e6d170"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49164
I20260709 12:18:06.743480  2021 raft_consensus.cc:493] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:06.743845  2021 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.747314  2176 consensus_queue.cc:1048] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.748476  2094 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:06.752739  2021 raft_consensus.cc:515] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.754179  2020 raft_consensus.cc:1240] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Rejecting Update request from peer 019d73dc41d44c29920aa7460204384d for earlier term 1. Current term is 2. Ops: []
I20260709 12:18:06.756315  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.756388  2021 leader_election.cc:290] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.757148  1945 raft_consensus.cc:3055] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.757526  1945 raft_consensus.cc:740] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.758365  1945 consensus_queue.cc:260] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.759296  2176 consensus_queue.cc:1059] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.759706  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.762442  2141 raft_consensus.cc:993] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.762912  2176 raft_consensus.cc:1081] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.764767  1945 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.765962  2019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49164
I20260709 12:18:06.769477  2019 raft_consensus.cc:493] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:06.769881  2019 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.775753  1945 raft_consensus.cc:2468] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:06.771845  2141 consensus_queue.cc:1048] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:18:06.776907  1987 leader_election.cc:304] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.777853  2177 raft_consensus.cc:2804] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.778343  2177 raft_consensus.cc:697] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.779175  2177 consensus_queue.cc:237] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.780355  2019 raft_consensus.cc:515] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.789945  2019 leader_election.cc:290] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.792441  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.791639  2094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e2f17f371f4255bfece699c9a8a378" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.793347  1945 raft_consensus.cc:3055] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.793865  1945 raft_consensus.cc:740] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.793964  2094 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.794953  1945 consensus_queue.cc:260] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.796645  1945 raft_consensus.cc:3060] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.799341  1843 catalog_manager.cc:5697] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 019d73dc41d44c29920aa7460204384d (127.21.150.193) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.802265  2094 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:06.803944  1985 leader_election.cc:304] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.804852  2177 raft_consensus.cc:2804] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.805361  2177 raft_consensus.cc:697] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.805513  1945 raft_consensus.cc:2468] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:06.806317  2177 consensus_queue.cc:237] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.813638  2151 raft_consensus.cc:493] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:06.814222  2151 raft_consensus.cc:515] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.816006  2151 leader_election.cc:290] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:06.815495  1843 catalog_manager.cc:5697] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 019d73dc41d44c29920aa7460204384d (127.21.150.193) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.816857  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:06.817360  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:06.817696  1945 raft_consensus.cc:2393] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76776f5079244e5db5b08abda16c7aef in current term 2: Already voted for candidate 6428ce197bfd4557b67da6a3eca8a0ca in this term.
I20260709 12:18:06.818943  2061 leader_election.cc:304] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76776f5079244e5db5b08abda16c7aef; no voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:06.819666  2151 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.826177  2151 raft_consensus.cc:2749] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:18:06.848281  2141 consensus_queue.cc:1048] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.854450  2177 consensus_queue.cc:1048] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.855352  2176 raft_consensus.cc:993] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.855872  2176 raft_consensus.cc:1081] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:06.858527  2020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49164
I20260709 12:18:06.860028  2020 raft_consensus.cc:493] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:06.860428  2020 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.861393  2176 consensus_queue.cc:1048] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.870330  2142 consensus_queue.cc:1048] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.872022  2020 raft_consensus.cc:515] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.878531  2020 leader_election.cc:290] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:06.878381  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:06.878423  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eb08e64f1954232a431fd84f8cbef6e" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.887084  1945 raft_consensus.cc:3055] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.887609  1945 raft_consensus.cc:740] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:06.888641  1945 consensus_queue.cc:260] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.889945  1945 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.901335  1945 raft_consensus.cc:2468] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:06.902585  1987 leader_election.cc:304] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:06.904959  2209 raft_consensus.cc:2804] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.918463  2177 consensus_queue.cc:1048] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.927971  2151 consensus_queue.cc:1048] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:06.937136  2213 consensus_queue.cc:1048] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.940670  2211 raft_consensus.cc:993] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: : Instructing follower 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:06.941231  2211 raft_consensus.cc:1081] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Signalling peer 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:06.945004  2093 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:51448
I20260709 12:18:06.945740  2093 raft_consensus.cc:493] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:06.946259  2093 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.954070  2209 raft_consensus.cc:697] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.956497  2093 raft_consensus.cc:515] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.956931  2209 consensus_queue.cc:237] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.960036  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:06.960742  1944 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.963660  2093 leader_election.cc:290] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:06.967456  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7057c1494f5445d9ad8b3a1d9a06f29" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:06.968080  2020 raft_consensus.cc:3055] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:06.968466  2020 raft_consensus.cc:740] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:06.969746  1841 catalog_manager.cc:5697] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 019d73dc41d44c29920aa7460204384d (127.21.150.193) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:06.969844  2020 consensus_queue.cc:260] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:06.972317  2020 raft_consensus.cc:3060] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:06.978677  2210 consensus_queue.cc:1048] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.981873  1944 raft_consensus.cc:2468] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 2.
I20260709 12:18:06.982961  2213 raft_consensus.cc:993] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: : Instructing follower 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:06.983457  2213 raft_consensus.cc:1081] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Signalling peer 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:06.984802  2020 raft_consensus.cc:2468] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 2.
I20260709 12:18:06.986646  2060 leader_election.cc:304] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:06.987531  2151 raft_consensus.cc:2804] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:06.988237  2151 raft_consensus.cc:697] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 2 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:06.995667  2143 consensus_queue.cc:1048] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.996592  2151 consensus_queue.cc:237] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.003197  2209 consensus_queue.cc:1048] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:06.988257  1945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:53984
I20260709 12:18:07.006282  1945 raft_consensus.cc:493] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.006740  1945 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.015017  1841 catalog_manager.cc:5697] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 1 to 2, leader changed from 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194) to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 2 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:07.018805  2093 raft_consensus.cc:1275] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.020880  2144 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.021852  1945 raft_consensus.cc:515] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.025785  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:07.026604  2019 raft_consensus.cc:3055] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.027012  2019 raft_consensus.cc:740] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:07.027702  2151 raft_consensus.cc:993] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:07.028254  2151 raft_consensus.cc:1081] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:07.028117  2019 consensus_queue.cc:260] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.031117  2019 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.039597  2020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49176
I20260709 12:18:07.040196  2021 raft_consensus.cc:1275] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.040328  2020 raft_consensus.cc:493] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.041044  2020 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.042649  2144 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.049873  2020 raft_consensus.cc:515] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.053406  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:07.053596  2020 leader_election.cc:290] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:07.053808  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e73bba6efab74c348ec9d818a055c266" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:07.054081  1944 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.053892  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ebabb8fe9df479bbac03224174cb2f9" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:07.054625  2095 raft_consensus.cc:3055] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.054759  2093 raft_consensus.cc:3060] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.055042  2095 raft_consensus.cc:740] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:07.055968  2095 consensus_queue.cc:260] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.056852  2095 raft_consensus.cc:3060] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.063377  2213 raft_consensus.cc:993] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: : Instructing follower 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:07.063951  2209 raft_consensus.cc:1081] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Signalling peer 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:07.064126  2093 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:07.064455  2095 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:07.065598  1910 leader_election.cc:304] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:07.065835  1985 leader_election.cc:304] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:07.066123  1943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:53984
I20260709 12:18:07.066784  1943 raft_consensus.cc:493] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:07.066839  2213 raft_consensus.cc:2804] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.067235  1943 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.067413  2213 raft_consensus.cc:697] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:07.067904  2144 raft_consensus.cc:2804] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.068199  1945 leader_election.cc:290] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:07.068619  2144 raft_consensus.cc:697] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:07.068663  2213 consensus_queue.cc:237] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.069591  2144 consensus_queue.cc:237] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.077901  2019 raft_consensus.cc:2468] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:07.079056  1944 raft_consensus.cc:2468] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:07.080163  1943 raft_consensus.cc:515] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.085114  1943 leader_election.cc:290] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:07.087060  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:07.087827  2095 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.089016  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "130a5f34a2e2482bbc722ba9759d76a1" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:07.089752  2019 raft_consensus.cc:3055] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:07.090157  2019 raft_consensus.cc:740] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:07.089870  1841 catalog_manager.cc:5697] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 1 to 2, leader changed from 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194) to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 2 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.091141  2019 consensus_queue.cc:260] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.092649  2019 raft_consensus.cc:3060] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.096613  2095 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:07.098063  1910 leader_election.cc:304] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:07.099337  2144 raft_consensus.cc:2804] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:07.099819  2144 raft_consensus.cc:697] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:07.100795  2019 raft_consensus.cc:2468] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:07.100718  2144 consensus_queue.cc:237] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:07.111296  1844 catalog_manager.cc:5697] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 1 to 2, leader changed from 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194) to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 2 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.112381  1841 catalog_manager.cc:5697] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 76776f5079244e5db5b08abda16c7aef (127.21.150.195) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:07.215229  1943 raft_consensus.cc:1275] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.218083  2213 consensus_queue.cc:1048] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.235473  2093 raft_consensus.cc:1275] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.238140  2213 consensus_queue.cc:1048] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.338447  1943 raft_consensus.cc:1275] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.339727  2213 consensus_queue.cc:1048] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.353412  1944 raft_consensus.cc:1275] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.355423  2093 raft_consensus.cc:1275] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.355692  2142 consensus_queue.cc:1048] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.358826  2142 consensus_queue.cc:1048] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:07.370409  2095 raft_consensus.cc:3060] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:07.387035  2095 raft_consensus.cc:1275] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.388564  2210 consensus_queue.cc:1048] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.406343  2019 raft_consensus.cc:1275] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.407938  2151 consensus_queue.cc:1048] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.419083  1943 raft_consensus.cc:1275] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.421097  2195 consensus_queue.cc:1048] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.515795  2095 raft_consensus.cc:1275] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.517294  2144 consensus_queue.cc:1048] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.527362  2019 raft_consensus.cc:1275] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.529289  2141 consensus_queue.cc:1048] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.533242  2095 raft_consensus.cc:1275] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.535037  2210 consensus_queue.cc:1048] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:07.541565  1943 raft_consensus.cc:1275] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.543074  2210 consensus_queue.cc:1048] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.578982  2095 raft_consensus.cc:1275] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.580519  2176 consensus_queue.cc:1048] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:07.590670  2019 raft_consensus.cc:1275] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:07.592535  2141 consensus_queue.cc:1048] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:09.715854  1841 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53186:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:18:09.718549  1841 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:18:09.784245  1935 tablet_service.cc:1511] Processing CreateTablet for tablet 79bd0739513441f5a582d008efee18a5 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:09.785455  2009 tablet_service.cc:1511] Processing CreateTablet for tablet 79bd0739513441f5a582d008efee18a5 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:09.786360  1935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79bd0739513441f5a582d008efee18a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.789914  2082 tablet_service.cc:1511] Processing CreateTablet for tablet 79bd0739513441f5a582d008efee18a5 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:18:09.789561  2009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79bd0739513441f5a582d008efee18a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.788995  1932 tablet_service.cc:1511] Processing CreateTablet for tablet 8268c64c204f43649d9e3efd325dec55 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:09.793051  2007 tablet_service.cc:1511] Processing CreateTablet for tablet 8268c64c204f43649d9e3efd325dec55 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:09.794020  1936 tablet_service.cc:1511] Processing CreateTablet for tablet a5dbb6ea5f514ab29d45302e2dbe4ded (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:09.794701  2007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8268c64c204f43649d9e3efd325dec55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.795622  1936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dbb6ea5f514ab29d45302e2dbe4ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.797086  2083 tablet_service.cc:1511] Processing CreateTablet for tablet a5dbb6ea5f514ab29d45302e2dbe4ded (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:09.798727  2084 tablet_service.cc:1511] Processing CreateTablet for tablet 8268c64c204f43649d9e3efd325dec55 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:18:09.791663  2082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79bd0739513441f5a582d008efee18a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.800698  2008 tablet_service.cc:1511] Processing CreateTablet for tablet a5dbb6ea5f514ab29d45302e2dbe4ded (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:18:09.802210  2008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dbb6ea5f514ab29d45302e2dbe4ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.802317  2084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8268c64c204f43649d9e3efd325dec55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.804466  2083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5dbb6ea5f514ab29d45302e2dbe4ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.807183  1932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8268c64c204f43649d9e3efd325dec55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.823406  1931 tablet_service.cc:1511] Processing CreateTablet for tablet 5138975548f74fbea94e127ae85df905 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:09.825752  1931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5138975548f74fbea94e127ae85df905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.825755  2011 tablet_service.cc:1511] Processing CreateTablet for tablet 5138975548f74fbea94e127ae85df905 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:09.828910  2011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5138975548f74fbea94e127ae85df905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.840843  2085 tablet_service.cc:1511] Processing CreateTablet for tablet 5138975548f74fbea94e127ae85df905 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:18:09.842935  2085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5138975548f74fbea94e127ae85df905. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.858953  1933 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6237d0f07c4726a4d13690f03d25d8 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:09.860028  2010 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6237d0f07c4726a4d13690f03d25d8 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:09.860836  1933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6237d0f07c4726a4d13690f03d25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.861918  2010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6237d0f07c4726a4d13690f03d25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.862273  2081 tablet_service.cc:1511] Processing CreateTablet for tablet 4e6237d0f07c4726a4d13690f03d25d8 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:18:09.863758  2081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e6237d0f07c4726a4d13690f03d25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.878715  1934 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:09.880234  1934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.879932  2006 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:09.883154  2006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.894594  2080 tablet_service.cc:1511] Processing CreateTablet for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:18:09.896190  2080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd6f02f8c2a42c383dc2ad25c8f2038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.913354  1930 tablet_service.cc:1511] Processing CreateTablet for tablet c1b2c06fd4a0493da5527db588f716bf (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:09.914268  2005 tablet_service.cc:1511] Processing CreateTablet for tablet c1b2c06fd4a0493da5527db588f716bf (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:09.915475  1930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1b2c06fd4a0493da5527db588f716bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.915828  2005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1b2c06fd4a0493da5527db588f716bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.925913  2079 tablet_service.cc:1511] Processing CreateTablet for tablet c1b2c06fd4a0493da5527db588f716bf (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:18:09.927596  2079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1b2c06fd4a0493da5527db588f716bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:09.950779  2236 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:09.953344  1929 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3cc63efd6246e48285280551f1b5b9 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:09.955650  1929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3cc63efd6246e48285280551f1b5b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:18:09.956431  1892 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:18:09.986199  2237 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:09.992129  2085 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3cc63efd6246e48285280551f1b5b9 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:09.994758  2011 tablet_service.cc:1511] Processing CreateTablet for tablet 7b3cc63efd6246e48285280551f1b5b9 (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:18:09.996464  2235 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:09.997931  2011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3cc63efd6246e48285280551f1b5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:10.001515  2006 tablet_service.cc:1511] Processing CreateTablet for tablet 909b865b4ba24c70bd9db1aa1aa33a1f (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:10.002213  2085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b3cc63efd6246e48285280551f1b5b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:10.003196  2006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b865b4ba24c70bd9db1aa1aa33a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:10.009495  2078 tablet_service.cc:1511] Processing CreateTablet for tablet 909b865b4ba24c70bd9db1aa1aa33a1f (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:10.011005  2078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b865b4ba24c70bd9db1aa1aa33a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:10.003204  2236 tablet_bootstrap.cc:654] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.024876  1934 tablet_service.cc:1511] Processing CreateTablet for tablet 909b865b4ba24c70bd9db1aa1aa33a1f (DEFAULT_TABLE table=soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:18:10.028574  1934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909b865b4ba24c70bd9db1aa1aa33a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:18:10.036010  2237 tablet_bootstrap.cc:654] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.043202  2235 tablet_bootstrap.cc:654] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.067718  2235 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.068403  2235 ts_tablet_manager.cc:1403] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.000s
I20260709 12:18:10.069383  2237 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.070127  2237 ts_tablet_manager.cc:1403] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.001s
I20260709 12:18:10.070339  2236 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.070981  2236 ts_tablet_manager.cc:1403] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.121s	user 0.014s	sys 0.000s
I20260709 12:18:10.072216  2235 raft_consensus.cc:359] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.072954  2237 raft_consensus.cc:359] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.073082  2235 raft_consensus.cc:385] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.073871  2237 raft_consensus.cc:385] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.073918  2235 raft_consensus.cc:740] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.074349  2237 raft_consensus.cc:740] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.075162  2237 consensus_queue.cc:260] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.074954  2235 consensus_queue.cc:260] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.074956  2236 raft_consensus.cc:359] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.076401  2236 raft_consensus.cc:385] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.076726  2236 raft_consensus.cc:740] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.077667  2236 consensus_queue.cc:260] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.079280  2237 ts_tablet_manager.cc:1434] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:18:10.080492  2237 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.085107  2235 ts_tablet_manager.cc:1434] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.006s
I20260709 12:18:10.086858  2235 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.087322  2237 tablet_bootstrap.cc:654] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.095203  2235 tablet_bootstrap.cc:654] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.104085  2236 ts_tablet_manager.cc:1434] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.032s	user 0.003s	sys 0.005s
I20260709 12:18:10.105933  2236 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.114732  2235 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.115335  2235 ts_tablet_manager.cc:1403] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.001s
I20260709 12:18:10.118515  2235 raft_consensus.cc:359] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.119325  2235 raft_consensus.cc:385] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.119611  2235 raft_consensus.cc:740] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.120601  2235 consensus_queue.cc:260] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.121272  2236 tablet_bootstrap.cc:654] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.122982  2235 ts_tablet_manager.cc:1434] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:18:10.123982  2235 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.131161  2235 tablet_bootstrap.cc:654] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.134742  2236 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.135016  2237 tablet_bootstrap.cc:492] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.135459  2236 ts_tablet_manager.cc:1403] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.000s
I20260709 12:18:10.135591  2237 ts_tablet_manager.cc:1403] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.018s
I20260709 12:18:10.142554  2237 raft_consensus.cc:359] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.143344  2237 raft_consensus.cc:385] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.143914  2237 raft_consensus.cc:740] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.143136  2236 raft_consensus.cc:359] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.144557  2236 raft_consensus.cc:385] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.144939  2236 raft_consensus.cc:740] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.144752  2237 consensus_queue.cc:260] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.145968  2236 consensus_queue.cc:260] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.147493  2237 ts_tablet_manager.cc:1434] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 12:18:10.148531  2236 ts_tablet_manager.cc:1434] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:18:10.149923  2236 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.154897  2237 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.156759  2236 tablet_bootstrap.cc:654] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.162631  2237 tablet_bootstrap.cc:654] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.163309  2235 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.163878  2235 ts_tablet_manager.cc:1403] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.008s
I20260709 12:18:10.171535  2236 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.172560  2236 ts_tablet_manager.cc:1403] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.012s
I20260709 12:18:10.172392  2235 raft_consensus.cc:359] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.173517  2235 raft_consensus.cc:385] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.173880  2235 raft_consensus.cc:740] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.174729  2235 consensus_queue.cc:260] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.176638  2213 raft_consensus.cc:493] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.177209  2235 ts_tablet_manager.cc:1434] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:18:10.177501  2237 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.177142  2213 raft_consensus.cc:515] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.178129  2237 ts_tablet_manager.cc:1403] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.004s
I20260709 12:18:10.178534  2235 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.180742  1943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.180872  2237 raft_consensus.cc:359] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.181716  2237 raft_consensus.cc:385] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.182070  2237 raft_consensus.cc:740] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.182771  2213 leader_election.cc:290] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.183005  2237 consensus_queue.cc:260] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.184793  2213 raft_consensus.cc:493] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.185278  2213 raft_consensus.cc:515] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.185381  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.186997  2093 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:10.187018  2235 tablet_bootstrap.cc:654] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.181721  1943 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:10.187798  2213 leader_election.cc:290] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.185554  2237 ts_tablet_manager.cc:1434] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:18:10.185811  2236 raft_consensus.cc:359] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.192073  2236 raft_consensus.cc:385] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.192216  1985 leader_election.cc:304] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.192358  2237 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.192489  2236 raft_consensus.cc:740] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.193801  2236 consensus_queue.cc:260] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.199060  2236 ts_tablet_manager.cc:1434] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.004s
I20260709 12:18:10.199170  1943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6237d0f07c4726a4d13690f03d25d8" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.200430  2236 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.200598  2237 tablet_bootstrap.cc:654] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.203912  2213 raft_consensus.cc:2804] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.204457  2213 raft_consensus.cc:493] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:18:10.208487  1987 leader_election.cc:343] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.209555  2213 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.210635  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6237d0f07c4726a4d13690f03d25d8" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.214286  2093 raft_consensus.cc:2468] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:10.216720  1985 leader_election.cc:304] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 019d73dc41d44c29920aa7460204384d
I20260709 12:18:10.219102  2248 raft_consensus.cc:2804] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.220427  2248 raft_consensus.cc:493] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.220906  2248 raft_consensus.cc:3060] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.220285  2236 tablet_bootstrap.cc:654] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.225030  2238 raft_consensus.cc:493] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.227937  2238 raft_consensus.cc:515] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.231284  2237 tablet_bootstrap.cc:492] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.232057  2237 ts_tablet_manager.cc:1403] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.010s
I20260709 12:18:10.233337  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.234957  2248 raft_consensus.cc:515] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.235116  2237 raft_consensus.cc:359] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
W20260709 12:18:10.235541  1911 leader_election.cc:343] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.235862  2237 raft_consensus.cc:385] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.236426  2237 raft_consensus.cc:740] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.237353  2237 consensus_queue.cc:260] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.237901  2213 raft_consensus.cc:515] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.238761  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6237d0f07c4726a4d13690f03d25d8" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:10.239099  1943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e6237d0f07c4726a4d13690f03d25d8" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.239408  2093 raft_consensus.cc:3060] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.239830  1943 raft_consensus.cc:3060] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.239985  2237 ts_tablet_manager.cc:1434] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:18:10.241218  2237 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.241272  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.242118  1944 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.245334  2238 leader_election.cc:290] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.245652  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
W20260709 12:18:10.248476  1910 leader_election.cc:343] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.249047  1910 leader_election.cc:304] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019d73dc41d44c29920aa7460204384d; no voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.249287  2237 tablet_bootstrap.cc:654] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.250126  2176 raft_consensus.cc:2749] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:10.246603  2235 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.251421  1944 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:10.251977  2235 ts_tablet_manager.cc:1403] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.074s	user 0.005s	sys 0.020s
I20260709 12:18:10.252733  1987 leader_election.cc:304] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:10.257649  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:10.258553  2095 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.260506  2235 raft_consensus.cc:359] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.264997  2235 raft_consensus.cc:385] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.265555  2235 raft_consensus.cc:740] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.250128  2093 raft_consensus.cc:2468] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:10.263602  1943 raft_consensus.cc:2468] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:10.269892  1985 leader_election.cc:304] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.272603  2213 leader_election.cc:290] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.273317  2213 raft_consensus.cc:2804] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.273788  2213 raft_consensus.cc:697] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:10.267177  2235 consensus_queue.cc:260] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.284087  2095 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:10.286283  2254 raft_consensus.cc:2804] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.290747  2254 raft_consensus.cc:697] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:10.292431  2254 consensus_queue.cc:237] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.288456  2213 consensus_queue.cc:237] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.288061  2248 leader_election.cc:290] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.297912  2236 tablet_bootstrap.cc:492] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.302004  2237 tablet_bootstrap.cc:492] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.302522  2236 ts_tablet_manager.cc:1403] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.103s	user 0.016s	sys 0.006s
I20260709 12:18:10.302580  2237 ts_tablet_manager.cc:1403] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.062s	user 0.028s	sys 0.031s
I20260709 12:18:10.305987  2236 raft_consensus.cc:359] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.306780  2236 raft_consensus.cc:385] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.307063  2236 raft_consensus.cc:740] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.307427  2237 raft_consensus.cc:359] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.308533  2237 raft_consensus.cc:385] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.308840  2237 raft_consensus.cc:740] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.309298  2236 consensus_queue.cc:260] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.309723  2237 consensus_queue.cc:260] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.312615  2237 ts_tablet_manager.cc:1434] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:18:10.314656  2235 ts_tablet_manager.cc:1434] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.062s	user 0.017s	sys 0.021s
I20260709 12:18:10.315171  2236 ts_tablet_manager.cc:1434] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:18:10.316275  2235 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.316303  2236 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.321847  2237 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.331099  2236 tablet_bootstrap.cc:654] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.332181  2237 tablet_bootstrap.cc:654] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.336817  2235 tablet_bootstrap.cc:654] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.342978  2236 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.344745  2236 ts_tablet_manager.cc:1403] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 12:18:10.348239  2236 raft_consensus.cc:359] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.349035  2236 raft_consensus.cc:385] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.351032  2237 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.351720  2237 ts_tablet_manager.cc:1403] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.010s
I20260709 12:18:10.353191  2235 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.353940  2235 ts_tablet_manager.cc:1403] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.004s
I20260709 12:18:10.354557  2237 raft_consensus.cc:359] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.355242  2237 raft_consensus.cc:385] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.349329  2236 raft_consensus.cc:740] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.355680  2237 raft_consensus.cc:740] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.356614  2237 consensus_queue.cc:260] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.356663  2236 consensus_queue.cc:260] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.357903  2235 raft_consensus.cc:359] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.358729  2235 raft_consensus.cc:385] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.359021  2235 raft_consensus.cc:740] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.359828  2237 ts_tablet_manager.cc:1434] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:18:10.359865  2235 consensus_queue.cc:260] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.359990  2236 ts_tablet_manager.cc:1434] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:18:10.362424  2236 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.362562  2235 ts_tablet_manager.cc:1434] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:18:10.364074  2235 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.370167  2236 tablet_bootstrap.cc:654] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.371522  2235 tablet_bootstrap.cc:654] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.396734  1844 catalog_manager.cc:5697] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.405356  2238 raft_consensus.cc:493] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.406076  2238 raft_consensus.cc:515] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.409807  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5138975548f74fbea94e127ae85df905" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.410483  2095 raft_consensus.cc:2468] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:10.411511  2238 leader_election.cc:290] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.412248  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5138975548f74fbea94e127ae85df905" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.412909  2237 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.413538  1910 leader_election.cc:304] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.414476  2213 raft_consensus.cc:493] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.414942  2238 raft_consensus.cc:2804] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.415432  2238 raft_consensus.cc:493] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.414983  2213 raft_consensus.cc:515] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.415802  2238 raft_consensus.cc:3060] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.417569  2213 leader_election.cc:290] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.412926  2019 raft_consensus.cc:2468] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:10.419320  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
W20260709 12:18:10.421494  1985 leader_election.cc:343] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.427714  2237 tablet_bootstrap.cc:654] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.429816  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.431521  2238 raft_consensus.cc:515] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
W20260709 12:18:10.434392  1987 leader_election.cc:343] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.434891  1987 leader_election.cc:304] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.435168  2238 leader_election.cc:290] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.436833  2213 raft_consensus.cc:2749] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:10.437217  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5138975548f74fbea94e127ae85df905" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:10.437301  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5138975548f74fbea94e127ae85df905" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.438055  2095 raft_consensus.cc:3060] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.438084  2019 raft_consensus.cc:3060] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.440608  2237 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.441195  2237 ts_tablet_manager.cc:1403] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.000s
I20260709 12:18:10.401139  1844 catalog_manager.cc:5697] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.444286  2236 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.444839  2236 ts_tablet_manager.cc:1403] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.024s
I20260709 12:18:10.444511  2237 raft_consensus.cc:359] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.445278  2237 raft_consensus.cc:385] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.445607  2237 raft_consensus.cc:740] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.446074  2019 raft_consensus.cc:2468] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:10.446345  2237 consensus_queue.cc:260] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.447595  1911 leader_election.cc:304] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:10.447865  2176 raft_consensus.cc:493] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.448534  2235 tablet_bootstrap.cc:492] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.448494  2176 raft_consensus.cc:515] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.448973  2237 ts_tablet_manager.cc:1434] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:18:10.449164  2235 ts_tablet_manager.cc:1403] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.086s	user 0.026s	sys 0.037s
I20260709 12:18:10.450730  2237 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.453054  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
W20260709 12:18:10.454466  1910 leader_election.cc:343] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.454551  2176 leader_election.cc:290] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.459895  2095 raft_consensus.cc:2468] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:10.455971  2236 raft_consensus.cc:359] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.461961  2236 raft_consensus.cc:385] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.460816  2237 tablet_bootstrap.cc:654] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.462428  2236 raft_consensus.cc:740] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.463624  2236 consensus_queue.cc:260] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.456156  2238 raft_consensus.cc:2804] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.453377  2235 raft_consensus.cc:359] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.466651  2235 raft_consensus.cc:385] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.466997  2235 raft_consensus.cc:740] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.454907  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.468919  2019 raft_consensus.cc:2468] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:10.470497  2238 raft_consensus.cc:697] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:10.470958  1911 leader_election.cc:304] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.472779  2176 raft_consensus.cc:2804] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.473275  2176 raft_consensus.cc:493] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.473766  2176 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.474432  2235 consensus_queue.cc:260] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.477190  2236 ts_tablet_manager.cc:1434] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.005s
I20260709 12:18:10.478574  2236 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.489759  2235 ts_tablet_manager.cc:1434] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 12:18:10.492105  2235 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.495694  2176 raft_consensus.cc:515] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.486228  2238 consensus_queue.cc:237] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.499181  2236 tablet_bootstrap.cc:654] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.507910  2237 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.508747  2237 ts_tablet_manager.cc:1403] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.030s
I20260709 12:18:10.509672  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
W20260709 12:18:10.512647  1910 leader_election.cc:343] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.513760  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.513965  2269 raft_consensus.cc:493] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.513880  2237 raft_consensus.cc:359] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.510252  2235 tablet_bootstrap.cc:654] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.508929  2176 leader_election.cc:290] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.528685  2237 raft_consensus.cc:385] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.528760  2269 raft_consensus.cc:515] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.529247  2237 raft_consensus.cc:740] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.514582  2020 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.530648  2237 consensus_queue.cc:260] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.533495  2269 leader_election.cc:290] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.532938  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dbb6ea5f514ab29d45302e2dbe4ded" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.540377  2237 ts_tablet_manager.cc:1434] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.031s	user 0.015s	sys 0.008s
I20260709 12:18:10.547574  1844 catalog_manager.cc:5697] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.550732  2237 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: Bootstrap starting.
I20260709 12:18:10.548765  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dbb6ea5f514ab29d45302e2dbe4ded" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.553776  2224 raft_consensus.cc:493] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.548792  2095 raft_consensus.cc:2468] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
I20260709 12:18:10.555469  2224 raft_consensus.cc:515] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.557004  1910 leader_election.cc:304] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.557858  2238 raft_consensus.cc:2804] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.558234  2238 raft_consensus.cc:493] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.558565  2238 raft_consensus.cc:3060] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.559504  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909b865b4ba24c70bd9db1aa1aa33a1f" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.560283  1944 raft_consensus.cc:2468] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.560617  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909b865b4ba24c70bd9db1aa1aa33a1f" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.561733  2061 leader_election.cc:304] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.552500  2019 raft_consensus.cc:2468] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 0.
W20260709 12:18:10.562430  2060 leader_election.cc:343] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.563025  2224 leader_election.cc:290] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.581949  2236 tablet_bootstrap.cc:492] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.582602  2236 ts_tablet_manager.cc:1403] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.009s
I20260709 12:18:10.579003  2224 raft_consensus.cc:2804] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.587268  2224 raft_consensus.cc:493] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.588181  2224 raft_consensus.cc:3060] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.588297  2236 raft_consensus.cc:359] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.591380  2273 raft_consensus.cc:493] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.591543  2275 raft_consensus.cc:493] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.589624  2236 raft_consensus.cc:385] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.593043  2236 raft_consensus.cc:740] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.592542  2273 raft_consensus.cc:515] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.594123  2020 raft_consensus.cc:2468] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:10.593976  2236 consensus_queue.cc:260] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.595804  2237 tablet_bootstrap.cc:654] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.595685  1911 leader_election.cc:304] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.596581  2236 ts_tablet_manager.cc:1434] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.002s
I20260709 12:18:10.596885  2269 raft_consensus.cc:2804] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.597527  2269 raft_consensus.cc:697] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:10.597817  2236 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.598489  2269 consensus_queue.cc:237] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.592626  2275 raft_consensus.cc:515] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.602818  2238 raft_consensus.cc:515] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.605702  2275 leader_election.cc:290] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.609512  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.610352  1944 raft_consensus.cc:2468] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.611909  2061 leader_election.cc:304] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.612563  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dbb6ea5f514ab29d45302e2dbe4ded" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.612797  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.613328  2019 raft_consensus.cc:3060] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.613824  2275 raft_consensus.cc:2804] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.614275  2275 raft_consensus.cc:493] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.614637  2275 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.615825  2020 raft_consensus.cc:2468] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.619269  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.619534  2021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
W20260709 12:18:10.623765  1910 leader_election.cc:343] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:18:10.626168  1911 leader_election.cc:343] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.624706  2238 leader_election.cc:290] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.626693  1911 leader_election.cc:304] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 019d73dc41d44c29920aa7460204384d; no voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.626827  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5dbb6ea5f514ab29d45302e2dbe4ded" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:10.627514  2095 raft_consensus.cc:3060] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.627516  2238 raft_consensus.cc:2749] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:10.631731  2236 tablet_bootstrap.cc:654] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.633703  2235 tablet_bootstrap.cc:492] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.634346  2235 ts_tablet_manager.cc:1403] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.143s	user 0.015s	sys 0.005s
I20260709 12:18:10.640132  2237 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: No bootstrap required, opened a new log
I20260709 12:18:10.640805  2237 ts_tablet_manager.cc:1403] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: Time spent bootstrapping tablet: real 0.090s	user 0.018s	sys 0.013s
I20260709 12:18:10.644311  2237 raft_consensus.cc:359] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.645133  2237 raft_consensus.cc:385] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.645483  2237 raft_consensus.cc:740] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.646237  2237 consensus_queue.cc:260] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.617772  2273 leader_election.cc:290] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.649914  2237 ts_tablet_manager.cc:1434] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.001s
I20260709 12:18:10.649721  2019 raft_consensus.cc:2468] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:10.652781  1911 leader_election.cc:304] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:10.653776  2238 raft_consensus.cc:2804] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.654320  2238 raft_consensus.cc:697] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:10.654628  2275 raft_consensus.cc:515] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.655184  2238 consensus_queue.cc:237] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.658320  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.658974  1944 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.659554  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.660338  2019 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.661336  2095 raft_consensus.cc:2468] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 1.
I20260709 12:18:10.660266  2235 raft_consensus.cc:359] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.667244  2235 raft_consensus.cc:385] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.667865  2235 raft_consensus.cc:740] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.669674  1944 raft_consensus.cc:2468] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:10.669694  2235 consensus_queue.cc:260] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.673344  2061 leader_election.cc:304] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.674805  2276 raft_consensus.cc:2804] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.676257  2235 ts_tablet_manager.cc:1434] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 12:18:10.678385  2235 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.679153  2236 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.679733  2236 ts_tablet_manager.cc:1403] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.082s	user 0.008s	sys 0.004s
I20260709 12:18:10.682659  2236 raft_consensus.cc:359] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.683363  2236 raft_consensus.cc:385] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.683701  2236 raft_consensus.cc:740] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.683490  1843 catalog_manager.cc:5697] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.684458  2236 consensus_queue.cc:260] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.686218  2235 tablet_bootstrap.cc:654] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.709380  2213 consensus_queue.cc:1048] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:10.695030  2224 raft_consensus.cc:515] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.715406  2224 leader_election.cc:290] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.716087  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909b865b4ba24c70bd9db1aa1aa33a1f" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.716917  2020 raft_consensus.cc:3060] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.718547  2236 ts_tablet_manager.cc:1434] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.010s
I20260709 12:18:10.719990  2236 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Bootstrap starting.
I20260709 12:18:10.720772  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909b865b4ba24c70bd9db1aa1aa33a1f" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.721414  1944 raft_consensus.cc:3060] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.726336  2254 consensus_queue.cc:1048] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:10.726524  2020 raft_consensus.cc:2468] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:10.732266  2236 tablet_bootstrap.cc:654] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.734292  2060 leader_election.cc:304] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.732407  2235 tablet_bootstrap.cc:492] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.735563  2282 raft_consensus.cc:2804] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.735918  2235 ts_tablet_manager.cc:1403] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.012s
I20260709 12:18:10.736636  2282 raft_consensus.cc:697] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:10.737866  2282 consensus_queue.cc:237] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.739423  2235 raft_consensus.cc:359] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.740316  2235 raft_consensus.cc:385] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.740658  2235 raft_consensus.cc:740] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.741623  2235 consensus_queue.cc:260] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.749284  2236 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: No bootstrap required, opened a new log
I20260709 12:18:10.749719  2235 ts_tablet_manager.cc:1434] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:18:10.750448  2236 ts_tablet_manager.cc:1403] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.001s
I20260709 12:18:10.751286  2235 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: Bootstrap starting.
I20260709 12:18:10.752157  2255 consensus_queue.cc:1048] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:10.756043  2257 consensus_queue.cc:1048] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:10.758996  2235 tablet_bootstrap.cc:654] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: Neither blocks nor log segments found. Creating new log.
I20260709 12:18:10.759631  2236 raft_consensus.cc:359] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.760406  2236 raft_consensus.cc:385] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.760690  2236 raft_consensus.cc:740] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.761458  2236 consensus_queue.cc:260] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.763821  2236 ts_tablet_manager.cc:1434] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:18:10.768357  2275 leader_election.cc:290] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.777653  2019 raft_consensus.cc:2468] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:10.779657  2276 raft_consensus.cc:697] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:10.781355  2276 consensus_queue.cc:237] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.788635  2281 raft_consensus.cc:493] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.789568  2281 raft_consensus.cc:515] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.824503  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.824790  2256 raft_consensus.cc:493] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.829777  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.807801  1944 raft_consensus.cc:2468] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
W20260709 12:18:10.834158  2060 leader_election.cc:343] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:18:10.836762  2281 leader_election.cc:290] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.831027  1945 raft_consensus.cc:2468] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.846717  2061 leader_election.cc:304] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:10.848722  2275 raft_consensus.cc:493] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:10.849113  2275 raft_consensus.cc:515] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.850366  2276 raft_consensus.cc:2804] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.851743  2276 raft_consensus.cc:493] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.854367  2276 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.854652  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:10.856129  1843 catalog_manager.cc:5697] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 0 to 1, leader changed from <none> to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 1 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.861881  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.863270  1945 raft_consensus.cc:2468] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.865232  2061 leader_election.cc:304] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.829542  2256 raft_consensus.cc:515] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.862553  2275 leader_election.cc:290] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.861505  1841 catalog_manager.cc:5697] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 0 to 1, leader changed from <none> to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 1 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.869103  2020 raft_consensus.cc:2468] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 0.
I20260709 12:18:10.873742  2281 raft_consensus.cc:2804] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.874233  2281 raft_consensus.cc:493] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.873591  2235 tablet_bootstrap.cc:492] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: No bootstrap required, opened a new log
I20260709 12:18:10.874604  2281 raft_consensus.cc:3060] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.874979  2235 ts_tablet_manager.cc:1403] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent bootstrapping tablet: real 0.124s	user 0.054s	sys 0.063s
I20260709 12:18:10.878353  2235 raft_consensus.cc:359] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.879164  2235 raft_consensus.cc:385] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:18:10.879504  2235 raft_consensus.cc:740] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Initialized, Role: FOLLOWER
I20260709 12:18:10.879779  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:10.880529  1945 raft_consensus.cc:2468] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 0.
I20260709 12:18:10.880249  2235 consensus_queue.cc:260] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.881711  1987 leader_election.cc:304] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:10.883759  2254 raft_consensus.cc:2804] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:18:10.884261  2254 raft_consensus.cc:493] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:18:10.884632  2254 raft_consensus.cc:3060] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.884832  2235 ts_tablet_manager.cc:1434] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:18:10.898124  2256 leader_election.cc:290] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.897307  2254 raft_consensus.cc:515] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.899267  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef" is_pre_election: true
I20260709 12:18:10.900942  2254 leader_election.cc:290] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:10.901113  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.902098  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:10.903334  1945 raft_consensus.cc:3060] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.904481  2276 raft_consensus.cc:515] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.908331  2276 leader_election.cc:290] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.909308  1944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.909407  2281 raft_consensus.cc:515] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.910113  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.910585  1944 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.911044  2020 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:10.911654  2093 raft_consensus.cc:2393] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6428ce197bfd4557b67da6a3eca8a0ca in current term 1: Already voted for candidate 76776f5079244e5db5b08abda16c7aef in this term.
I20260709 12:18:10.913779  2281 leader_election.cc:290] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:10.915575  1943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:10.918054  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1b2c06fd4a0493da5527db588f716bf" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:10.919170  2019 raft_consensus.cc:2393] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 76776f5079244e5db5b08abda16c7aef in current term 1: Already voted for candidate 6428ce197bfd4557b67da6a3eca8a0ca in this term.
I20260709 12:18:10.920372  2060 leader_election.cc:304] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76776f5079244e5db5b08abda16c7aef; no voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:10.921645  2281 raft_consensus.cc:2749] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:18:10.925202  2020 raft_consensus.cc:2468] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:10.926762  2060 leader_election.cc:304] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:10.928696  2281 raft_consensus.cc:2804] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.927904  1841 catalog_manager.cc:5697] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 0 to 1, leader changed from <none> to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 1 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.928936  1945 raft_consensus.cc:2468] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 1.
I20260709 12:18:10.930176  2281 raft_consensus.cc:697] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:10.931922  1944 raft_consensus.cc:2468] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 1.
I20260709 12:18:10.931851  2281 consensus_queue.cc:237] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.933480  1987 leader_election.cc:304] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:10.935236  2254 raft_consensus.cc:2804] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:18:10.935963  2254 raft_consensus.cc:697] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:10.937232  2254 consensus_queue.cc:237] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:10.946993  1844 catalog_manager.cc:5697] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 0 to 1, leader changed from <none> to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 1 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:10.947304  1841 catalog_manager.cc:5697] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 1 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:10.983011  2020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79bd0739513441f5a582d008efee18a5"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
mode: GRACEFUL
new_leader_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:49178
I20260709 12:18:10.983678  2020 raft_consensus.cc:606] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Received request to transfer leadership to TS 019d73dc41d44c29920aa7460204384d
I20260709 12:18:10.990516  2093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
mode: GRACEFUL
new_leader_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:18:10.991138  2093 raft_consensus.cc:606] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Received request to transfer leadership to TS 019d73dc41d44c29920aa7460204384d
I20260709 12:18:11.008074  1945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:53986
I20260709 12:18:11.008782  1945 raft_consensus.cc:606] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:11.017911  2281 raft_consensus.cc:493] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:18:11.018052  2093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
mode: GRACEFUL
new_leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:51458
I20260709 12:18:11.018764  2093 raft_consensus.cc:606] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Received request to transfer leadership to TS 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:11.018630  2281 raft_consensus.cc:515] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.021179  2281 leader_election.cc:290] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:11.021871  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "019d73dc41d44c29920aa7460204384d" is_pre_election: true
I20260709 12:18:11.022343  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" is_pre_election: true
I20260709 12:18:11.023252  2020 raft_consensus.cc:2393] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76776f5079244e5db5b08abda16c7aef in current term 1: Already voted for candidate 019d73dc41d44c29920aa7460204384d in this term.
I20260709 12:18:11.024286  2060 leader_election.cc:304] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76776f5079244e5db5b08abda16c7aef; no voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:11.025074  2281 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:18:11.031445  2281 raft_consensus.cc:2749] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:18:11.035830  2238 consensus_queue.cc:1048] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.042896  2238 consensus_queue.cc:1048] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.049201  2273 consensus_queue.cc:1048] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.063263  2176 consensus_queue.cc:1048] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.157590  2238 consensus_queue.cc:1048] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:11.161916  2276 consensus_queue.cc:1048] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.168784  2176 raft_consensus.cc:993] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:11.169324  2273 raft_consensus.cc:1081] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:11.183879  2276 consensus_queue.cc:1048] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.185464  2019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49164
I20260709 12:18:11.190389  2019 raft_consensus.cc:493] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.190928  2019 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.193092  2269 consensus_queue.cc:1048] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:11.197671  2256 raft_consensus.cc:993] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: : Instructing follower 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:11.198268  2257 raft_consensus.cc:1081] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Signalling peer 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:11.200398  1945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79bd0739513441f5a582d008efee18a5"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:53984
I20260709 12:18:11.201468  1945 raft_consensus.cc:493] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.202064  1945 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.208595  2019 raft_consensus.cc:515] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.215474  1945 raft_consensus.cc:515] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.218664  2018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:11.219313  2018 raft_consensus.cc:3055] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.219641  2018 raft_consensus.cc:740] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:11.220238  2093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79bd0739513441f5a582d008efee18a5" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:11.220927  2093 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.220589  2018 consensus_queue.cc:260] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.221841  2018 raft_consensus.cc:3060] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.227155  1943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:11.228199  1943 raft_consensus.cc:3055] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.228681  1943 raft_consensus.cc:740] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:11.229403  2093 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:11.231396  2018 raft_consensus.cc:2468] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:11.231938  1910 leader_election.cc:304] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:11.233260  1945 leader_election.cc:290] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:11.233291  2273 raft_consensus.cc:2804] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:11.234169  2273 raft_consensus.cc:697] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:11.235191  2273 consensus_queue.cc:237] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.229898  1943 consensus_queue.cc:260] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.237545  1943 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.245765  1943 raft_consensus.cc:2468] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:11.245901  2019 leader_election.cc:290] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:11.248059  1987 leader_election.cc:304] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:11.248915  2256 raft_consensus.cc:2804] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:11.249409  2256 raft_consensus.cc:697] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:11.250242  2256 consensus_queue.cc:237] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.252488  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fd6f02f8c2a42c383dc2ad25c8f2038" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:11.253155  2095 raft_consensus.cc:3060] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.252707  1843 catalog_manager.cc:5697] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 1 to 2, leader changed from 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194) to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 2 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:11.254182  2276 consensus_queue.cc:1048] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.261341  2095 raft_consensus.cc:2468] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:11.266866  1841 catalog_manager.cc:5697] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 019d73dc41d44c29920aa7460204384d (127.21.150.193) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:11.267611  2275 raft_consensus.cc:993] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: : Instructing follower 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:11.268144  2275 raft_consensus.cc:1081] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Signalling peer 019d73dc41d44c29920aa7460204384d to start an election
I20260709 12:18:11.270174  1945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
 from {username='slave'} at 127.0.0.1:53978
I20260709 12:18:11.271091  1945 raft_consensus.cc:493] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.271399  1945 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.278024  2275 consensus_queue.cc:1048] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:18:11.284878  1945 raft_consensus.cc:515] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.295433  1945 leader_election.cc:290] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:11.295923  2019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:11.298555  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8268c64c204f43649d9e3efd325dec55" candidate_uuid: "019d73dc41d44c29920aa7460204384d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:11.299160  2095 raft_consensus.cc:3055] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.299480  2095 raft_consensus.cc:740] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:11.300268  2095 consensus_queue.cc:260] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.301239  2095 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.308348  2095 raft_consensus.cc:2468] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 019d73dc41d44c29920aa7460204384d in term 2.
I20260709 12:18:11.309509  1910 leader_election.cc:304] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 6428ce197bfd4557b67da6a3eca8a0ca
I20260709 12:18:11.310333  2273 raft_consensus.cc:2804] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:11.310838  2273 raft_consensus.cc:697] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Leader. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:11.311657  2273 consensus_queue.cc:237] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.319921  1843 catalog_manager.cc:5697] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d reported cstate change: term changed from 1 to 2, leader changed from 76776f5079244e5db5b08abda16c7aef (127.21.150.195) to 019d73dc41d44c29920aa7460204384d (127.21.150.193). New cstate: current_term: 2 leader_uuid: "019d73dc41d44c29920aa7460204384d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:11.341549  2275 consensus_queue.cc:1048] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.352231  2276 consensus_queue.cc:1048] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.359401  2275 raft_consensus.cc:993] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: : Instructing follower 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:11.359830  2276 raft_consensus.cc:1081] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Signalling peer 6428ce197bfd4557b67da6a3eca8a0ca to start an election
I20260709 12:18:11.361513  2018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
 from {username='slave'} at 127.0.0.1:49176
I20260709 12:18:11.361936  2018 raft_consensus.cc:493] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:11.362180  2018 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.368571  2018 raft_consensus.cc:515] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.371033  2018 leader_election.cc:290] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 76776f5079244e5db5b08abda16c7aef (127.21.150.195:41069)
I20260709 12:18:11.371877  1945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:11.372581  1945 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.372599  2095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b3cc63efd6246e48285280551f1b5b9" candidate_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "76776f5079244e5db5b08abda16c7aef"
I20260709 12:18:11.373270  2095 raft_consensus.cc:3055] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:18:11.373615  2095 raft_consensus.cc:740] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:11.374356  2095 consensus_queue.cc:260] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.375566  2095 raft_consensus.cc:3060] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.379873  1945 raft_consensus.cc:2468] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:11.381187  1987 leader_election.cc:304] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 6428ce197bfd4557b67da6a3eca8a0ca; no voters: 
I20260709 12:18:11.381973  2256 raft_consensus.cc:2804] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:18:11.381918  2095 raft_consensus.cc:2468] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6428ce197bfd4557b67da6a3eca8a0ca in term 2.
I20260709 12:18:11.382711  2256 raft_consensus.cc:697] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Leader. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:11.383550  2256 consensus_queue.cc:237] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:11.390728  1844 catalog_manager.cc:5697] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca reported cstate change: term changed from 1 to 2, leader changed from 76776f5079244e5db5b08abda16c7aef (127.21.150.195) to 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194). New cstate: current_term: 2 leader_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:11.436704  2256 consensus_queue.cc:1048] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.451042  2213 consensus_queue.cc:1048] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:18:11.705662  2095 raft_consensus.cc:1275] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.707542  2269 consensus_queue.cc:1048] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:11.716531  2018 raft_consensus.cc:1275] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.718689  2273 consensus_queue.cc:1048] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:11.767445  1945 raft_consensus.cc:1275] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.768086  2095 raft_consensus.cc:1275] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.769264  2254 consensus_queue.cc:1048] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:11.770340  2213 consensus_queue.cc:1048] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:11.797717  1945 raft_consensus.cc:1275] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.799078  2254 consensus_queue.cc:1048] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:11.810933  2095 raft_consensus.cc:1275] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 6428ce197bfd4557b67da6a3eca8a0ca: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.812877  2256 consensus_queue.cc:1048] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:11.851663  2095 raft_consensus.cc:1275] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.853325  2273 consensus_queue.cc:1048] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:18:11.862094  2018 raft_consensus.cc:3060] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:18:11.870095  2018 raft_consensus.cc:1275] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Refusing update from remote peer 019d73dc41d44c29920aa7460204384d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:18:11.871516  2238 consensus_queue.cc:1048] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:18:14.026646  1844 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53186:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:18:14.051508  1844 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53186:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:18:14.090493 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:18:14.200078  2330 log.cc:927] Time spent T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:18:14.272748  2356 tablet.cc:1724] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:18:14.300956  1910 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.150.195:41069, user_credentials={real_user=slave}} blocked reactor thread for 97818.5us
W20260709 12:18:14.307165  2348 log.cc:927] Time spent T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
I20260709 12:18:14.324023  2364 tablet.cc:1724] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.341483  2358 tablet.cc:1724] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.352223  2357 tablet.cc:1724] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.367316  2375 tablet.cc:1724] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.387554  2372 tablet.cc:1724] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.398089  2368 tablet.cc:1724] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.403672  2374 tablet.cc:1724] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.457549  2376 tablet.cc:1724] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.474608  2377 tablet.cc:1724] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.433656  2356 tablet.cc:1724] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.526588  2372 tablet.cc:1724] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.448324  2373 tablet.cc:1724] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.531291  2358 tablet.cc:1724] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.563109  1946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d"
dest_uuid: "019d73dc41d44c29920aa7460204384d"
mode: GRACEFUL
new_leader_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:34830
I20260709 12:18:14.563902  1946 raft_consensus.cc:606] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Received request to transfer leadership to TS 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:14.518606  2383 tablet.cc:1724] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.603973  2375 tablet.cc:1724] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.577800  2364 tablet.cc:1724] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.621501  2377 tablet.cc:1724] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.631932  2376 tablet.cc:1724] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.635861  2358 tablet.cc:1724] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.637344  2357 tablet.cc:1724] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.650151  2373 tablet.cc:1724] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.653513  2368 tablet.cc:1724] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.654170  2020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f64a2d787148489595e36eba30e6d170"
dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
mode: GRACEFUL
new_leader_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:56586
I20260709 12:18:14.656111  2020 raft_consensus.cc:606] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Received request to transfer leadership to TS 76776f5079244e5db5b08abda16c7aef
I20260709 12:18:14.658397 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:18:14.659027 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:14.654788  2374 tablet.cc:1724] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.671836  2376 tablet.cc:1724] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.678869  2383 tablet.cc:1724] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.685043  2356 tablet.cc:1724] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:18:14.745399  1844 catalog_manager.cc:6242] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: soft_deleted_table [id=89ea65e8046649af9387bf229d8bbd1d] alter complete (version 1)
I20260709 12:18:15.012943  2354 raft_consensus.cc:993] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: : Instructing follower 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:15.013521  2360 raft_consensus.cc:1081] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Signalling peer 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:15.015290  2091 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:51440
I20260709 12:18:15.015810  2091 raft_consensus.cc:493] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:15.016129  2091 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.024919  2091 raft_consensus.cc:515] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:15.027916  2020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:15.028614  2020 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.029946  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b3515fe56a41a0a60a492a7304bb9d" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:15.028023  2091 leader_election.cc:290] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 3 election: Requested vote from peers 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215), 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695)
I20260709 12:18:15.030603  1946 raft_consensus.cc:3055] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:15.031114  1946 raft_consensus.cc:740] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 019d73dc41d44c29920aa7460204384d, State: Running, Role: LEADER
I20260709 12:18:15.032204  1946 consensus_queue.cc:260] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:15.033905  1946 raft_consensus.cc:3060] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.037637  2020 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 3.
I20260709 12:18:15.038911  2060 leader_election.cc:304] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6428ce197bfd4557b67da6a3eca8a0ca, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:15.039906  2363 raft_consensus.cc:2804] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:15.041559  2363 raft_consensus.cc:697] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 3 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:15.041731  1946 raft_consensus.cc:2468] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 3.
I20260709 12:18:15.042629  2363 consensus_queue.cc:237] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } }
I20260709 12:18:15.057026  1842 catalog_manager.cc:5697] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 2 to 3, leader changed from 019d73dc41d44c29920aa7460204384d (127.21.150.193) to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 3 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } }
I20260709 12:18:15.193424  2366 raft_consensus.cc:993] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: : Instructing follower 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:15.194106  2254 raft_consensus.cc:1081] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Signalling peer 76776f5079244e5db5b08abda16c7aef to start an election
I20260709 12:18:15.195851  2091 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f64a2d787148489595e36eba30e6d170"
dest_uuid: "76776f5079244e5db5b08abda16c7aef"
 from {username='slave'} at 127.0.0.1:51448
I20260709 12:18:15.196619  2091 raft_consensus.cc:493] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:18:15.196978  2091 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.204381  2091 raft_consensus.cc:515] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:15.206678  2091 leader_election.cc:290] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 3 election: Requested vote from peers 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695), 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215)
I20260709 12:18:15.207551  1946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "019d73dc41d44c29920aa7460204384d"
I20260709 12:18:15.207764  2018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f64a2d787148489595e36eba30e6d170" candidate_uuid: "76776f5079244e5db5b08abda16c7aef" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6428ce197bfd4557b67da6a3eca8a0ca"
I20260709 12:18:15.208380  1946 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.208472  2018 raft_consensus.cc:3055] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:18:15.208931  2018 raft_consensus.cc:740] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6428ce197bfd4557b67da6a3eca8a0ca, State: Running, Role: LEADER
I20260709 12:18:15.209687  2018 consensus_queue.cc:260] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:15.211002  2018 raft_consensus.cc:3060] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:18:15.215384  1946 raft_consensus.cc:2468] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 3.
I20260709 12:18:15.216763  2061 leader_election.cc:304] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 019d73dc41d44c29920aa7460204384d, 76776f5079244e5db5b08abda16c7aef; no voters: 
I20260709 12:18:15.217602  2363 raft_consensus.cc:2804] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:18:15.218091  2363 raft_consensus.cc:697] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 3 LEADER]: Becoming Leader. State: Replica: 76776f5079244e5db5b08abda16c7aef, State: Running, Role: LEADER
I20260709 12:18:15.218861  2018 raft_consensus.cc:2468] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76776f5079244e5db5b08abda16c7aef in term 3.
I20260709 12:18:15.218989  2363 consensus_queue.cc:237] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } }
I20260709 12:18:15.234341  1842 catalog_manager.cc:5697] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef reported cstate change: term changed from 2 to 3, leader changed from 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194) to 76776f5079244e5db5b08abda16c7aef (127.21.150.195). New cstate: current_term: 3 leader_uuid: "76776f5079244e5db5b08abda16c7aef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76776f5079244e5db5b08abda16c7aef" member_type: VOTER last_known_addr { host: "127.21.150.195" port: 41069 } health_report { overall_health: HEALTHY } } }
I20260709 12:18:15.576267  2020 raft_consensus.cc:1275] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.577754  2363 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:15.590036  1946 raft_consensus.cc:1275] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.591771  2363 consensus_queue.cc:1048] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:15.660351 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:15.666361 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:15.666757 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:15.696815  1946 raft_consensus.cc:1275] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.698805  2363 consensus_queue.cc:1048] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "019d73dc41d44c29920aa7460204384d" member_type: VOTER last_known_addr { host: "127.21.150.193" port: 45695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:15.708931  2020 raft_consensus.cc:1275] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Refusing update from remote peer 76776f5079244e5db5b08abda16c7aef: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:18:15.710551  2389 consensus_queue.cc:1048] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6428ce197bfd4557b67da6a3eca8a0ca" member_type: VOTER last_known_addr { host: "127.21.150.194" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:18:16.667748 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:16.672991 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:16.673450 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:17.674763 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:17.682682 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:17.683089 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:18.684202 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:18.689563 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:18.689922 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:19.690933 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:19.695797 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:19.696198 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:20.697232 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:20.702961 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:20.703480 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:21.704555 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:21.710362 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:21.710774 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:22.711776 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:22.716738 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:22.717101 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:23.718133 22107 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:18:23.723587 22107 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:18:23.723979 22107 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:18:24.790269 22107 tablet_server.cc:179] TabletServer@127.21.150.193:0 shutting down...
W20260709 12:18:24.802431  1987 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
W20260709 12:18:24.806768  1987 consensus_peers.cc:597] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:24.836126 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:24.837394 22107 tablet_replica.cc:333] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.838925 22107 raft_consensus.cc:2243] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.839970 22107 raft_consensus.cc:2272] T 7b3cc63efd6246e48285280551f1b5b9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.843207 22107 tablet_replica.cc:333] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.844029 22107 raft_consensus.cc:2243] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.844659 22107 raft_consensus.cc:2272] T 909b865b4ba24c70bd9db1aa1aa33a1f P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.847275 22107 tablet_replica.cc:333] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.847973 22107 raft_consensus.cc:2243] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.848623 22107 raft_consensus.cc:2272] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.851996 22107 tablet_replica.cc:333] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.852921 22107 raft_consensus.cc:2243] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.853799 22107 raft_consensus.cc:2272] T c1b2c06fd4a0493da5527db588f716bf P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:24.854816  1987 consensus_peers.cc:597] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:24.856870 22107 tablet_replica.cc:333] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.857728 22107 raft_consensus.cc:2243] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.858726 22107 raft_consensus.cc:2272] T 5138975548f74fbea94e127ae85df905 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.861261 22107 tablet_replica.cc:333] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.861958 22107 raft_consensus.cc:2243] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.862882 22107 raft_consensus.cc:2272] T 8268c64c204f43649d9e3efd325dec55 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.865250 22107 tablet_replica.cc:333] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.866017 22107 raft_consensus.cc:2243] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.866572 22107 raft_consensus.cc:2272] T 99e2f17f371f4255bfece699c9a8a378 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.869251 22107 tablet_replica.cc:333] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.869858 22107 raft_consensus.cc:2243] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.870766 22107 raft_consensus.cc:2272] T 79bd0739513441f5a582d008efee18a5 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.873200 22107 tablet_replica.cc:333] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.873806 22107 raft_consensus.cc:2243] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.874754 22107 raft_consensus.cc:2272] T 5ebabb8fe9df479bbac03224174cb2f9 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.877295 22107 tablet_replica.cc:333] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.877929 22107 raft_consensus.cc:2243] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.878841 22107 raft_consensus.cc:2272] T a5dbb6ea5f514ab29d45302e2dbe4ded P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.881482 22107 tablet_replica.cc:333] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.882030 22107 raft_consensus.cc:2243] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.882962 22107 raft_consensus.cc:2272] T 130a5f34a2e2482bbc722ba9759d76a1 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.885406 22107 tablet_replica.cc:333] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.885953 22107 raft_consensus.cc:2243] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:24.887496 22107 raft_consensus.cc:2272] T 7e680db9e7f1422aaf06275944c0db3a P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:24.890645  1987 consensus_peers.cc:597] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:24.891207 22107 tablet_replica.cc:333] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.891896 22107 raft_consensus.cc:2243] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.892503 22107 raft_consensus.cc:2272] T f64a2d787148489595e36eba30e6d170 P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.894969 22107 tablet_replica.cc:333] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.895572 22107 raft_consensus.cc:2243] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.896135 22107 raft_consensus.cc:2272] T d7057c1494f5445d9ad8b3a1d9a06f29 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.899780 22107 tablet_replica.cc:333] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.900463 22107 raft_consensus.cc:2243] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.900952 22107 raft_consensus.cc:2272] T e73bba6efab74c348ec9d818a055c266 P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.903381 22107 tablet_replica.cc:333] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.903970 22107 raft_consensus.cc:2243] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.904544 22107 raft_consensus.cc:2272] T 4e6237d0f07c4726a4d13690f03d25d8 P 019d73dc41d44c29920aa7460204384d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.907092 22107 tablet_replica.cc:333] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.907724 22107 raft_consensus.cc:2243] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.908308 22107 raft_consensus.cc:2272] T 89b3515fe56a41a0a60a492a7304bb9d P 019d73dc41d44c29920aa7460204384d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:24.910982 22107 tablet_replica.cc:333] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d: stopping tablet replica
I20260709 12:18:24.911558 22107 raft_consensus.cc:2243] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:24.912127 22107 raft_consensus.cc:2272] T 0eb08e64f1954232a431fd84f8cbef6e P 019d73dc41d44c29920aa7460204384d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:24.924100  1987 consensus_peers.cc:597] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:18:24.927706  2061 consensus_peers.cc:597] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:24.963806 22107 tablet_server.cc:196] TabletServer@127.21.150.193:0 shutdown complete.
I20260709 12:18:25.008414 22107 tablet_server.cc:179] TabletServer@127.21.150.194:0 shutting down...
W20260709 12:18:25.038328  2060 consensus_peers.cc:597] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef -> Peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Couldn't send request to peer 6428ce197bfd4557b67da6a3eca8a0ca. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.042893 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:25.043941 22107 tablet_replica.cc:333] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.044844 22107 raft_consensus.cc:2243] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:18:25.045890  2061 consensus_peers.cc:597] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.045892 22107 raft_consensus.cc:2272] T c1b2c06fd4a0493da5527db588f716bf P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.048803 22107 tablet_replica.cc:333] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.049902 22107 raft_consensus.cc:2243] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.050796 22107 raft_consensus.cc:2272] T 7b3cc63efd6246e48285280551f1b5b9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.053267 22107 tablet_replica.cc:333] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.053894 22107 raft_consensus.cc:2243] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.054512 22107 raft_consensus.cc:2272] T 909b865b4ba24c70bd9db1aa1aa33a1f P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.057015 22107 tablet_replica.cc:333] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.057794 22107 raft_consensus.cc:2243] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.058727 22107 raft_consensus.cc:2272] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.061399 22107 tablet_replica.cc:333] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.062095 22107 raft_consensus.cc:2243] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.062630 22107 raft_consensus.cc:2272] T 8268c64c204f43649d9e3efd325dec55 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.064870 22107 tablet_replica.cc:333] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.065538 22107 raft_consensus.cc:2243] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.066071 22107 raft_consensus.cc:2272] T f64a2d787148489595e36eba30e6d170 P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.068574 22107 tablet_replica.cc:333] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.069157 22107 raft_consensus.cc:2243] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.070137 22107 raft_consensus.cc:2272] T 99e2f17f371f4255bfece699c9a8a378 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.072552 22107 tablet_replica.cc:333] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.073120 22107 raft_consensus.cc:2243] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.073735 22107 raft_consensus.cc:2272] T 79bd0739513441f5a582d008efee18a5 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.075961 22107 tablet_replica.cc:333] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.076508 22107 raft_consensus.cc:2243] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.077039 22107 raft_consensus.cc:2272] T a5dbb6ea5f514ab29d45302e2dbe4ded P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.079628 22107 tablet_replica.cc:333] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.080324 22107 raft_consensus.cc:2243] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.080880 22107 raft_consensus.cc:2272] T 5ebabb8fe9df479bbac03224174cb2f9 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.083249 22107 tablet_replica.cc:333] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.083832 22107 raft_consensus.cc:2243] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.084349 22107 raft_consensus.cc:2272] T 130a5f34a2e2482bbc722ba9759d76a1 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.086642 22107 tablet_replica.cc:333] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.087270 22107 raft_consensus.cc:2243] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.087809 22107 raft_consensus.cc:2272] T 5138975548f74fbea94e127ae85df905 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.090132 22107 tablet_replica.cc:333] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.090657 22107 raft_consensus.cc:2243] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.091123 22107 raft_consensus.cc:2272] T 7e680db9e7f1422aaf06275944c0db3a P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.093369 22107 tablet_replica.cc:333] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.093973 22107 raft_consensus.cc:2243] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.094870 22107 raft_consensus.cc:2272] T e73bba6efab74c348ec9d818a055c266 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.097388 22107 tablet_replica.cc:333] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.098006 22107 raft_consensus.cc:2243] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.098553 22107 raft_consensus.cc:2272] T d7057c1494f5445d9ad8b3a1d9a06f29 P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.101809 22107 tablet_replica.cc:333] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
W20260709 12:18:25.101895  2061 consensus_peers.cc:597] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.102428 22107 raft_consensus.cc:2243] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.103422 22107 raft_consensus.cc:2272] T 4e6237d0f07c4726a4d13690f03d25d8 P 6428ce197bfd4557b67da6a3eca8a0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.105820 22107 tablet_replica.cc:333] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.106623 22107 raft_consensus.cc:2243] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.107192 22107 raft_consensus.cc:2272] T 89b3515fe56a41a0a60a492a7304bb9d P 6428ce197bfd4557b67da6a3eca8a0ca [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.109642 22107 tablet_replica.cc:333] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca: stopping tablet replica
I20260709 12:18:25.110188 22107 raft_consensus.cc:2243] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.111109 22107 raft_consensus.cc:2272] T 0eb08e64f1954232a431fd84f8cbef6e P 6428ce197bfd4557b67da6a3eca8a0ca [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:25.131088  2061 consensus_peers.cc:597] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef -> Peer 019d73dc41d44c29920aa7460204384d (127.21.150.193:45695): Couldn't send request to peer 019d73dc41d44c29920aa7460204384d. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.193:45695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.157997 22107 tablet_server.cc:196] TabletServer@127.21.150.194:0 shutdown complete.
I20260709 12:18:25.201762 22107 tablet_server.cc:179] TabletServer@127.21.150.195:0 shutting down...
I20260709 12:18:25.235324 22107 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:18:25.236340 22107 tablet_replica.cc:333] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.237555 22107 raft_consensus.cc:2243] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.238174 22107 raft_consensus.cc:2272] T c1b2c06fd4a0493da5527db588f716bf P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.241405 22107 tablet_replica.cc:333] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.242410 22107 raft_consensus.cc:2243] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.243085 22107 raft_consensus.cc:2272] T 2fd6f02f8c2a42c383dc2ad25c8f2038 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:25.246701  2060 consensus_peers.cc:597] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef -> Peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Couldn't send request to peer 6428ce197bfd4557b67da6a3eca8a0ca. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:35215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.246713 22107 tablet_replica.cc:333] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.247932 22107 raft_consensus.cc:2243] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.248703 22107 raft_consensus.cc:2272] T 7b3cc63efd6246e48285280551f1b5b9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.251961 22107 tablet_replica.cc:333] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.252795 22107 raft_consensus.cc:2243] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.254025 22107 raft_consensus.cc:2272] T 909b865b4ba24c70bd9db1aa1aa33a1f P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.257130 22107 tablet_replica.cc:333] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.257977 22107 raft_consensus.cc:2243] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.258615 22107 raft_consensus.cc:2272] T 8268c64c204f43649d9e3efd325dec55 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.262007 22107 tablet_replica.cc:333] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.262969 22107 raft_consensus.cc:2243] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 3 LEADER]: Raft consensus shutting down.
W20260709 12:18:25.263950  2060 consensus_peers.cc:597] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef -> Peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Couldn't send request to peer 6428ce197bfd4557b67da6a3eca8a0ca. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:35215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.264163 22107 raft_consensus.cc:2272] T f64a2d787148489595e36eba30e6d170 P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.267222 22107 tablet_replica.cc:333] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.267851 22107 raft_consensus.cc:2243] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.268374 22107 raft_consensus.cc:2272] T 5138975548f74fbea94e127ae85df905 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.270707 22107 tablet_replica.cc:333] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.271250 22107 raft_consensus.cc:2243] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.271694 22107 raft_consensus.cc:2272] T 7e680db9e7f1422aaf06275944c0db3a P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.274462 22107 tablet_replica.cc:333] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.275398 22107 raft_consensus.cc:2243] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.276129 22107 raft_consensus.cc:2272] T a5dbb6ea5f514ab29d45302e2dbe4ded P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.278831 22107 tablet_replica.cc:333] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.279553 22107 raft_consensus.cc:2243] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.280246 22107 raft_consensus.cc:2272] T 130a5f34a2e2482bbc722ba9759d76a1 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:18:25.280494  2060 consensus_peers.cc:597] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef -> Peer 6428ce197bfd4557b67da6a3eca8a0ca (127.21.150.194:35215): Couldn't send request to peer 6428ce197bfd4557b67da6a3eca8a0ca. Status: Network error: Client connection negotiation failed: client connection to 127.21.150.194:35215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:18:25.282817 22107 tablet_replica.cc:333] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.283537 22107 raft_consensus.cc:2243] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.284055 22107 raft_consensus.cc:2272] T 5ebabb8fe9df479bbac03224174cb2f9 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.286382 22107 tablet_replica.cc:333] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.286991 22107 raft_consensus.cc:2243] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.287536 22107 raft_consensus.cc:2272] T 0eb08e64f1954232a431fd84f8cbef6e P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.289896 22107 tablet_replica.cc:333] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.290479 22107 raft_consensus.cc:2243] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.291031 22107 raft_consensus.cc:2272] T 79bd0739513441f5a582d008efee18a5 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.293248 22107 tablet_replica.cc:333] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.293906 22107 raft_consensus.cc:2243] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.294436 22107 raft_consensus.cc:2272] T 99e2f17f371f4255bfece699c9a8a378 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.296631 22107 tablet_replica.cc:333] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.297147 22107 raft_consensus.cc:2243] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.298116 22107 raft_consensus.cc:2272] T d7057c1494f5445d9ad8b3a1d9a06f29 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.300395 22107 tablet_replica.cc:333] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.300912 22107 raft_consensus.cc:2243] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.301391 22107 raft_consensus.cc:2272] T e73bba6efab74c348ec9d818a055c266 P 76776f5079244e5db5b08abda16c7aef [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.303527 22107 tablet_replica.cc:333] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.304021 22107 raft_consensus.cc:2243] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:18:25.304576 22107 raft_consensus.cc:2272] T 4e6237d0f07c4726a4d13690f03d25d8 P 76776f5079244e5db5b08abda16c7aef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.306650 22107 tablet_replica.cc:333] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef: stopping tablet replica
I20260709 12:18:25.307109 22107 raft_consensus.cc:2243] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.308036 22107 raft_consensus.cc:2272] T 89b3515fe56a41a0a60a492a7304bb9d P 76776f5079244e5db5b08abda16c7aef [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.350831 22107 tablet_server.cc:196] TabletServer@127.21.150.195:0 shutdown complete.
I20260709 12:18:25.391110 22107 master.cc:562] Master@127.21.150.254:38955 shutting down...
I20260709 12:18:25.428040 22107 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:18:25.428792 22107 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:18:25.429283 22107 tablet_replica.cc:333] T 00000000000000000000000000000000 P d503b778e4fb47f9b81a48377edb6025: stopping tablet replica
I20260709 12:18:25.452452 22107 master.cc:584] Master@127.21.150.254:38955 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20799 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (41439 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (400999 ms total)
[  PASSED  ] 8 tests.
I20260709 12:18:25.513362 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2228 messages since previous log ~32 seconds ago
I20260709 12:18:25.513664 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2225 messages since previous log ~32 seconds ago
I20260709 12:18:25.513859 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2223 messages since previous log ~32 seconds ago
I20260709 12:18:25.514079 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~56 seconds ago
I20260709 12:18:25.514283 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~56 seconds ago
I20260709 12:18:25.514520 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 11 messages since previous log ~187 seconds ago
I20260709 12:18:25.514802 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:18:25.515120 22107 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
